Free 17 search & AI crawlers · Score + prioritised fixes
AI search readiness checker: can ChatGPT, Perplexity and Google AI use your page?
Paste a URL to see which search and AI crawlers can reach the page, whether the HTML is readable without JavaScript, and what to fix first.
AI search readiness results
Shared report. These results were saved when the check ran. Run it again below for live results, or test your own site.
· AI readiness
- to fix
- warnings
- passed
Fix these first ranked by impact
Search & AI crawler access
What your robots.txt tells each crawler about this URL. Keep search and AI-search crawlers allowed; blocking training crawlers is your choice.
| Crawler | Type | robots.txt |
|---|---|---|
/100
Want to be cited in AI answers, not just crawled? I'll review your content, entities and technical setup on a free 30-minute call.
Book a strategy callWhat this AI readiness check tells you
An AI-ready page is one that AI search systems are allowed to crawl, can read in the raw HTML, and can quote without guessing who wrote it or when. This checker tests all three. It reads your robots.txt the way each crawler does, requests the page as a real fetcher would, and scores five areas: access, readable content, answer-ready structure, entity and trust signals, and basic speed. Every failed check comes with a specific fix, and the most important ones come with copy-paste code.
It is honest about what it can’t see: it doesn’t ask ChatGPT or Perplexity whether they cite you today (that varies by prompt and changes daily). It checks the things you control that decide whether you can be cited.
How to use it
- Paste the URL of a page you want to appear in AI answers: a service page, guide or product page usually matters more than the home page.
- Read the Fix these first list. It is ranked by impact, so blocked crawlers and noindex come before nice-to-haves.
- Check the crawler table. Search and AI-search crawlers should say Allowed; training crawlers are your choice.
- Fix, then run the check again. Share the report link with your developer so they see exactly what failed.
What each check means
Access: robots.txt, firewalls and snippet controls
AI companies run separate crawlers for different jobs. OAI-SearchBot, PerplexityBot and Claude-SearchBot build the indexes used for AI search answers and citations, so blocking them removes you from those answers. GPTBot, ClaudeBot, CCBot and the Google-Extended and Applebot-Extended tokens relate to model training; blocking them is a business decision and does not remove you from AI search. User-initiated fetchers such as ChatGPT-User and Perplexity-User visit when someone shares your link; OpenAI and Perplexity say robots.txt may not apply to these.
Google’s AI Overviews and AI Mode use the normal Googlebot index, so a page must be indexable and allowed to show a snippet. noindex, nosnippet or max-snippet:0 keep it out. The checker also requests the page as OAI-SearchBot and PerplexityBot: if your CDN or firewall returns 403 to them while normal visitors get 200, you’re blocked even though robots.txt says yes. noai and noimageai meta tags are non-standard, and the major AI companies don’t document support for them.
Readable content: is it in the HTML?
Google renders JavaScript. Most AI crawlers don’t: they read the HTML your server sends. If your text only appears after a JavaScript app loads, those crawlers see an empty shell. The tool counts the words inside <main>/<article> in the raw response and flags app shells such as an empty #root or #__next.
Answer-ready structure
AI systems quote passages, not pages. Question-style headings followed by a direct answer, a concise opening paragraph, lists and tables, and a visible “updated” date all make a passage easier to lift accurately. These are good writing habits, not tricks, so they carry a lower weight than access.
Entity and trust signals
Organization or Person schema with sameAs links to your official profiles, a named author and linked About and Contact pages help any system work out who stands behind the content. Google says no special schema is required for its AI features, so treat this as clarity for every reader, human or machine.
llms.txt
The tool reports whether /llms.txt exists, for information only. Google’s documentation says Search doesn’t use llms.txt and the file neither helps nor hurts your visibility, and no major AI search engine has confirmed using it. It never affects your score.
Frequently asked questions
Should I block GPTBot?
Blocking GPTBot keeps your future content out of OpenAI model training. It does not remove you from ChatGPT search, which uses OAI-SearchBot. Many publishers block training crawlers and allow search crawlers; the right choice depends on your business model. The robots.txt generator builds either policy.
Does a high score guarantee I’ll be cited by ChatGPT or Google AI Overviews?
No. The score measures whether you can be crawled, read and quoted. Whether you are cited depends on relevance, authority and competition for each question. A low score, though, is a reliable sign you are losing citations you could win.
Why does my page pass in a browser but fail the HTML check?
Your browser runs JavaScript; most AI crawlers don’t. Server-side rendering or static generation puts the content in the first HTML response. WordPress themes render content on the server, so this usually affects JavaScript frameworks and some page builders.
Do I need an llms.txt file?
Not for Google. It’s an optional proposal; add one if you like, but spend your time on crawler access and clear content first.
How is this different from the free SEO audit?
The free SEO audit covers classic ranking factors across 30+ checks. This tool focuses on AI search: crawler-by-crawler access, firewall blocks, snippet controls and answer structure. Run both for a complete picture.
Want help getting cited in AI answers? See my SEO and AI search services or book a free strategy call.
Let’s talk
Find your biggest growth lever
Tell me where you are and where you want to be. I’ll reply with an honest view of what it takes, usually within one business day.
- Free 30-minute strategy call
- Your site reviewed before we speak
- No obligation, no hard sell
Thank you, message received.