What is AI crawlers?
User-agents operated by AI companies that crawl the web to populate their search indexes and answer queries in real time. Major ones include OpenAI's GPTBot and ChatGPT-User, Anthropic's ClaudeBot, Perplexity's PerplexityBot, and Google's Google-Extended.
Each AI crawler can be allowed or blocked individually via robots.txt. Sites that want to be cited in AI search results should explicitly allow these crawlers — blocking them removes the site from AI search visibility entirely.
How this shows up in your audit
Every Growth Blueprint includes an AI search visibility section that looks at how your store shows up in ChatGPT, Perplexity, Gemini, and Microsoft Copilot answers — and what's keeping it out of them. You get specific, non-technical instructions for improving how AI assistants see and cite your store.