ClaudeBot
Anthropic · Collects for training
Blocking ClaudeBot tells Anthropic to leave your site's future content out of its model training data. Anthropic supports Crawl-delay for it, and warns that blocking its IP addresses instead of using robots.txt may not work, because that stops it reading the file.
Fact sheet
- Operator
- Anthropic
- What it is for
- It collects text that is used to train AI models.
- Assistant it feeds
- Does not feed any assistant's answers
- Counts towards the Parsigo score
- No. What it collects is used to train models, and training does not decide whether you are cited.
- Official documentation
- support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler
- Official IP ranges
- JSON list
If you block it
You ask it to stop collecting your content. It changes neither whether you are cited nor your score.
Allow or block
A group of its own makes ClaudeBot ignore the “User-agent: *” rules. If you have Disallow lines there that should still apply to it, repeat them inside its group. To allow it, writing nothing is usually enough.
Allow
User-agent: ClaudeBot Allow: /
Block
User-agent: ClaudeBot Disallow: /
Test it on your site
We run the diagnostic on your site and show you whether ClaudeBot can get in, with the robots.txt line that decides it. Free, no sign-up.