There are really only two causes
Every "why am I missing" listicle fans out into five or seven items, but they all collapse into two layers that decide a citation, and a page can ace one and still lose on the other. Cause A is whether the AI crawlers can fetch your page at all. Cause B is whether any source those engines already trust mentions your brand by name. These are the two layers behind AI search visibility, and the reason a generic list is unhelpful is that it never tells you which layer is failing for you. The fix for A is nothing like the fix for B, so guessing wastes weeks.
The reflex is to blame Cause A first, usually robots.txt. It is the wrong default. Most invisible B2B brands have a perfectly crawlable site and are missing for Cause B. We have the data on that below, but the practical point comes first: do not start fixing the technical line until you have confirmed it is actually the problem.
The 60-second test that tells you which one is yours
Ask for your own homepage the way an AI crawler would, and read the live response rather than your robots.txt file. One request settles it:
curl -A "GPTBot" -I https://yourdomain.com/
If you get back a normal page (an HTTP 200), the crawler is getting in and your problem is almost certainly Cause B, the citation footprint, not the technical line everyone blames first. If you get a 403, a redirect to a challenge, or a block page, you have a Cause A problem, and it is more often a CDN or WAF bot rule than anything in robots.txt. Either way, do one careful check: a single curl can be a fluke, so confirm with a repeat and against a real page, not just the homepage. The AI crawler readiness guide and the AI crawler checklist cover the full version, including the rendering traps a curl alone will not catch.
One precision point on the bots. GPTBot and ClaudeBot are training crawlers, so blocking them mainly keeps you out of model training, not out of live answers. The bots that decide whether an engine can cite you in a live answer are different ones: OAI-SearchBot and ChatGPT-User for OpenAI, Claude-SearchBot and Claude-User for Anthropic, and PerplexityBot for Perplexity. GPTBot is still a fine quick proxy, because a blanket bot rule usually blocks all of them at once, but if you want to be citable, make sure robots.txt allows the search and fetch bots, not just the training crawlers you happen to recognize.
Why it is usually the footprint, with the data
This is the part incumbents assert and never prove. We measured it. In our Invisible 10 study, we ran 600 model responses across the four largest AI engines (ChatGPT, Claude, Gemini, and Perplexity) against ten funded mid-market compliance vendors with crawlable sites. None of the ten was cited once. The same 600 answers named other companies instead: TalentLMS 107 times, LogicGate 73, AuditBoard 69.
Being real, funded, and technically clean did not buy a single mention. The engines favored other names because those names had the footprint, the comparison posts, review-site entries, analyst notes, and community threads the engines had already read. That is the decisive cause for most invisible brands, and it is the one a robots.txt check will never surface.
Now route your cause to its fix
Once the self-test has sorted you, the work splits cleanly.
If it is Cause A (crawler blocked): this is a configuration fix, and it lands in days. Work through the AI crawler checklist to clear the block, then re-run the test to confirm the engines can read you.
If it is Cause B (no trusted source names you): the job is building a citation footprint, which is slower and depends on other people's pages. Start with AI search visibility for the full picture, and with answer engine optimization for making your own pages easy for an engine to lift and attribute. Then confirm progress over repeated checks, because a single look cannot tell a real gain from week-to-week noise. Our guide on measuring AI citations covers the metrics that matter.
One adjacent question: "is something broken, or is this just slow?" If the crawler is in and you are simply early, you are probably not broken, just waiting on the footprint clock. How long that takes has its own answer in how long it takes to show up in AI search.
Common questions
Why does ChatGPT name my competitors but not me?
Because the engine is repeating the sources it has read, and those sources name them, not you. It is rarely a quality judgment about your product. In our Invisible 10 study, 600 model responses across the four largest AI engines cited zero of ten funded compliance vendors, while naming TalentLMS 107 times, LogicGate 73, and AuditBoard 69. The cited brands had a deeper third-party footprint the engines could draw on. That is the gap to close, and it is the most common cause.
I rank on the first page of Google. Why am I invisible in ChatGPT and Perplexity?
Because they read different signals. Google ranks a list of links; AI engines assemble an answer from the sources they trust and name brands inside it. A crawlable, well-ranked site helps but does not guarantee a citation, because the engine still has to find a trusted source that mentions you by name. Strong Google rankings and an empty AI citation footprint commonly sit side by side.
Is it just my robots.txt blocking GPTBot?
Usually not, and it is the first thing to over-blame. Run the 60-second test: request your own homepage with a GPTBot user agent and see what comes back. If you get your page, the crawler is in and the cause is almost certainly your citation footprint, not the technical line. If you get a 403 or a challenge, the block is more often a CDN or WAF bot rule than a robots.txt entry. The AI crawler checklist walks through confirming it.
How long until this is fixed once I act on it?
It depends which cause is yours. The technical floor is a configuration change that lands in days. The citation footprint takes weeks to months, because it depends on other people's pages, not yours. The timeline question has its own answer page.
Is something broken, or is this just normal?
For most B2B brands that have never invested in AI search visibility, being absent is the default, not a fault. Nothing has to be broken for you to be missing. The engines simply favored other names because other names had the footprint. Confirm your cause with the self-test, then fix the one that applies.
How Web Cited helps
The self-test gives you your bucket in a minute. If you want it confirmed without the manual work, the free 10-minute AI search audit walks five buyer prompts across the main engines so you can see for yourself whether you are named. The Free Snapshot checks one buyer prompt against your domain and returns a homepage fix list. The full SXO Audit runs 25 buyer prompts across six engines with three trials each, plus the technical-floor checks, so you get both causes scored in one place and know exactly which one is yours.
Try the Free Snapshot See the SXO Audit
By the Web Cited Editorial Research Team. Last updated 1 June 2026.