Meta-ExternalFetcher
Meta · Another search engine or assistant
Meta-ExternalFetcher fetches individual links at a user's request. Meta also uses it to evaluate and improve agentic AI, for example to help AI navigate websites and complete tasks for people.
Fact sheet
- Operator
- Meta
- What it is for
- It fetches a page when someone asks it to, for an assistant the score does not measure.
- Assistant it feeds
- None of the assistants we measure
- Counts towards the Parsigo score
- No. It crawls for search or answers, but not for any of the assistants we measure.
- Official documentation
- developers.facebook.com/documentation/sharing/webmasters/web-crawlers
If you block it
You ask it not to fetch your pages when someone asks it to. Your score does not change: it belongs to none of the assistants we measure. The operator warns that robots.txt may not apply to these requests, because a person starts them.
Allow or block
A group of its own makes Meta-ExternalFetcher ignore the “User-agent: *” rules. If you have Disallow lines there that should still apply to it, repeat them inside its group. To allow it, writing nothing is usually enough.
Allow
User-agent: Meta-ExternalFetcher Allow: /
Block
User-agent: Meta-ExternalFetcher Disallow: /
Test it on your site
We run the diagnostic on your site and show you whether Meta-ExternalFetcher can get in, with the robots.txt line that decides it. Free, no sign-up.
Worth knowing
Fetches the page when someone requests it inside a Meta app. Meta warns it may bypass robots.txt because it acts on a person's request.