- AI crawlers
- Google-Extended
Google-Extended
Google · Opt-out signal
Google-Extended never shows up in your logs: Google crawls with its usual user agents and reads this name in robots.txt only as an instruction. Besides training, it covers grounding: the Search content that Gemini Apps and Vertex AI give the model when it answers.
Fact sheet
- Operator
- What it is for
- It tells the operator what it may do with what its other bots have already crawled: train models and, for Google, use it in Gemini's answers.
- Assistant it feeds
- Not a crawler: a name in robots.txt that limits how AI uses your content.
- Counts towards the Parsigo score
- No. Opt-out signals do not count towards the score.
- Official documentation
- developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers
If you block it
You limit what AI does with your content without leaving search. Your score does not change.
Allow or block
Google-Extended is not a crawler: only Disallow has any effect, and blocking it does not take you out of search.
Block
User-agent: Google-Extended Disallow: /
Test it on your site
We run the diagnostic on your site and show you whether Google-Extended can get in, with the robots.txt line that decides it. Free, no sign-up.
Worth knowing
Controls whether Google may use your content to train Gemini and for grounding (content from its index handed to the model at answer time) in Gemini Apps and Vertex AI. It does not affect AI Overviews or search, and it is not a ranking signal.