How to check if ChatGPT can see your website (5 tests, 10 minutes)

6 min read

ChatGPT search finds and cites pages through OpenAI's own crawlers plus third-party search indexes. If any link in that chain is broken for your site, you won't be cited, however good your content is. These five tests find the usual culprits, in order of how often they're the problem.

Fast path: paste a URL into the AI visibility checker. It runs tests 1–3 for ChatGPT, Perplexity, Claude and other engines at once and suggests fixes. The manual versions are below.

Test 1: Is OpenAI's search crawler allowed in robots.txt?

Open https://yoursite.com/robots.txt and look for these names:

• OAI-SearchBot: indexes pages for ChatGPT search. Must be allowed to be cited.
• ChatGPT-User: fetches a page when a user asks ChatGPT about it. Should be allowed.
• GPTBot: training only. Blocking it doesn't affect citations.

Watch for Disallow: / under a group that names OAI-SearchBot or ChatGPT-User, often copied from a "block all AI" list. Also check User-agent: *: if it disallows your pages and OpenAI's bots have no group of their own, they're blocked too. The robots.txt tester shows the exact deciding line for OAI-SearchBot.

Test 2: Does your CDN or firewall let it through?

robots.txt can say yes while the edge says no. Look in your CDN/WAF for:

• An "AI bots", "AI scrapers" or "bot fight" setting that blocks or challenges AI crawlers.
• Firewall events where the user agent contains OAI-SearchBot or ChatGPT-User and the action is block or challenge.
• Rate limits or country blocks that would catch crawler traffic (OpenAI publishes its crawler IP ranges, so you can allowlist them).

A rough self-test from a terminal: curl -sI -A "OAI-SearchBot/1.0; +https://openai.com/searchbot" https://yoursite.com/page. A 403 or a challenge page here means a user-agent rule is blocking it. A 200 is encouraging, but not proof, because some CDNs verify real bots by IP.

Test 3: Is your content in the HTML?

OpenAI's crawlers, like most AI crawlers, read the HTML your server sends and don't run JavaScript. Pick a sentence from the middle of your page and run:

curl -s https://yoursite.com/page | grep -c "a sentence from your page"

0 means the text is added by JavaScript, and to ChatGPT the page is mostly empty. Fix it with server-side rendering or static generation for content pages.

Test 4: Is the page indexable at all?

• No noindex in the meta robots tag or the X-Robots-Tag header (easy to miss).
• The canonical points to this page on your production host (common mistakes).
• The page is in Bing's index: search site:yoursite.com/page on Bing. Many AI search products, including Microsoft Copilot, rely on Bing, so verify your site in Bing Webmaster Tools and submit your sitemap (or use IndexNow).

The free SEO config check covers the first two in one go.

Test 5: Ask ChatGPT directly

With web search on, ask questions where your page is the best answer, for example "What does [your product] cost?" or "[your brand] vs [competitor]". Then:

• Check the sources ChatGPT lists. Is your page among them?
• Paste your URL and ask ChatGPT to summarise it. If it says it can't access the page, test 1, 2 or 3 is failing for ChatGPT-User.

Results vary between runs and by location, so treat this as a spot check, not a measurement.

Keep it working

Most AI-visibility breakages aren't decisions. They're side effects: a security plugin update, a "block bots" toggle someone flipped during a scraping incident, or a robots.txt copied from another project. They're silent, and no Search Console report covers them.