- AI crawlers
- Twitterbot
Twitterbot
X · Social network
Twitterbot reads the card markup of a page shared on X: if the page is blocked no card appears, and if only the image is blocked the card has no thumbnail. The content stays cached for 7 days after the link is posted.
Fact sheet
- Operator
- X
- What it is for
- It reads your page when someone shares a link, to show the title, description and image.
- Assistant it feeds
- Does not feed any assistant's answers
- Counts towards the Parsigo score
- No. It is a social network bot, not an AI one.
If you block it
When someone shares a link to your site, it may show without a title or image. Your score does not change.
Allow or block
A group of its own makes Twitterbot ignore the “User-agent: *” rules. If you have Disallow lines there that should still apply to it, repeat them inside its group. To allow it, writing nothing is usually enough.
Allow
User-agent: Twitterbot Allow: /
Block
User-agent: Twitterbot Disallow: /
Test it on your site
We run the diagnostic on your site and show you whether Twitterbot can get in, with the robots.txt line that decides it. Free, no sign-up.