AI2Bot
Allen Institute for AI · Collects for training
The Allen Institute for AI uses what AI2Bot finds to train open language models. Its page does not mention robots.txt; it says you can use the bot's user agent string to filter or reject its traffic.
Fact sheet
- Operator
- Allen Institute for AI
- What it is for
- It collects text that is used to train AI models.
- Assistant it feeds
- Does not feed any assistant's answers
- Counts towards the Parsigo score
- No. What it collects is used to train models, and training does not decide whether you are cited.
- Official documentation
- allenai.org/crawler
If you block it
You ask it to stop collecting your content. It changes neither whether you are cited nor your score.
Allow or block
A group of its own makes AI2Bot ignore the “User-agent: *” rules. If you have Disallow lines there that should still apply to it, repeat them inside its group. To allow it, writing nothing is usually enough.
Allow
User-agent: AI2Bot Allow: /
Block
User-agent: AI2Bot Disallow: /
Test it on your site
We run the diagnostic on your site and show you whether AI2Bot can get in, with the robots.txt line that decides it. Free, no sign-up.