AI Crawler Checker

Being mentioned by ChatGPT, Perplexity, or Gemini has a precondition nobody checks until something goes wrong: the AI has to actually be able to read your site. A firewall rule, a CDN setting, or a single line in robots.txt can quietly block every AI crawler without breaking anything else — your site still loads fine for a human visitor, so there is nothing to notice. The first sign is usually just silence: zero mentions, week after week, with no obvious reason why.

The AI Crawler Checker answers that question directly instead of leaving you to guess. It sends a real HTTP request to your site using the exact user-agent string of GPTBot, ClaudeBot, PerplexityBot, and seven other AI and search crawlers, then shows you exactly how your server responded to each one: allowed, blocked, rate-limited, or unreachable. That is a meaningfully different check from reading your robots.txt file, which only tells you what a config file claims should happen. A CDN or bot-management rule can block a crawler that robots.txt says is welcome, and that gap is exactly what this catches.

Alongside the live crawler test, the page also reports your robots.txt rules for AI bots, whether your homepage carries a noindex tag that hides it from search entirely, and whether you have an llms.txt file — the emerging standard some AI systems use to understand what your site is about. Every result is explained in plain language, with no HTTP status codes to decode.

Run it once after launch, and again any time you change hosting, add a CDN, or install a new security plugin — those are the moments a block is most likely to appear.