Bytespider
ByteDance · Another search engine or assistant
ByteDance documents Bytespider in Chinese, on its Toutiao Search webmaster platform, as that search engine's crawler, and explains there how it matches robots.txt rules. The page lists its IP ranges, says a genuine one's reverse DNS ends in bytedance.com, and does not mention AI training.
Fact sheet
- Operator
- ByteDance
- What it is for
- It crawls for a search engine or assistant the score does not measure.
- Assistant it feeds
- None of the assistants we measure
- Counts towards the Parsigo score
- No. It crawls for search or answers, but not for any of the assistants we measure.
- Official documentation
- zhanzhang.toutiao.com/docs/intro/26899
If you block it
You ask for your pages to be left out of the service it crawls for. Your score does not change: it belongs to none of the assistants we measure.
Allow or block
A group of its own makes Bytespider ignore the “User-agent: *” rules. If you have Disallow lines there that should still apply to it, repeat them inside its group. To allow it, writing nothing is usually enough.
Allow
User-agent: Bytespider Allow: /
Block
User-agent: Bytespider Disallow: /
Test it on your site
We run the diagnostic on your site and show you whether Bytespider can get in, with the robots.txt line that decides it. Free, no sign-up.