Crawlers & AI bots
Which bots and AI crawlers can access your site?
Search engines, ChatGPT, Claude, Perplexity, Gemini, Common Crawl, social networks and SEO tools: robots.txt checked bot by bot, plus a live firewall test.
Search engines
Google, Bing, Qwant, DuckDuckGo, Yandex, Baidu, Applebot.
AI search
ChatGPT Search, Claude, Perplexity, Le Chat, Duck.ai: to get cited in their answers.
AI training
GPTBot, ClaudeBot, Google-Extended, Applebot-Extended, Meta, Common Crawl, Bytespider.
Firewall test
We visit the page as each crawler to detect CDN blocking.
robots.txt analysis (RFC 9309) and live test · 30-minute cache
More free tools
Global Website RankingsTranco rank, history and competitors
SEO CheckerTags, content, links and indexing
Website Speed TestCore Web Vitals on mobile and desktop
Website Security CheckerDNSSEC, SPF, DMARC, certificate, headers
E-E-A-T Trust CheckerExperience, expertise, authority, trust
Domain Authority CheckerAuthority score and comparison