Do I need an llms.txt file?

llms.txt is a proposal, introduced in 2024, for a markdown file at the root of your site - /llms.txt - that points a language model at a curated, clean version of your most important content. The idea borrows its shape from robots.txt and sitemap.xml: a single predictable file an AI tool could read to understand your site without wading through navigation, ads, and markup. It is a reasonable idea on paper.

Its status is what matters here. llms.txt is a community proposal that the major AI companies never adopted. Publishing one does not opt you into anything official, because there is no official program to opt into. That distinction is the whole reason the honest answer to "do I need it" is "probably not yet," rather than "yes, it is table stakes."

For the engines that decide whether you are cited - ChatGPT, Claude, Gemini, Perplexity - there is no confirmed signal that an llms.txt file changes anything. None of them has stated that the file influences citation, and the crawler evidence cuts the other way: log studies across large numbers of sites find that the AI search bots overwhelmingly fetch your HTML directly and request llms.txt only a vanishingly small share of the time.

Google has been the most direct. It has publicly said it does not use llms.txt and has no plans to, and its search advocate John Mueller compared the file to the long-abandoned keywords meta tag - an unverifiable claim a site makes about its own content - while pointing out that server logs show the bots do not even fetch it. You will see confident blog claims that this or that engine "observably honors" llms.txt; treat those with caution, because correlation in a few case studies is not the same as a provider confirming the mechanism, and so far none has.

There is a real use case, and it is worth naming precisely so you can tell whether it is yours. Coding and IDE agents, and the MCP tools that pull documentation into an assistant, increasingly look for an llms.txt or llms-full.txt when they are pointed at a documentation site. For that audience the file does what it was designed to do: hands a tool a clean, navigable version of your docs instead of making it parse a rendered site. Several developer-documentation providers publish one for exactly this reason.

If you run developer docs that people consume through coding assistants, an llms.txt can be a small, real win. If your goal is getting your marketing or product pages cited in consumer AI search answers, that is a different audience with different mechanics, and this file is not how you reach it.

If the goal is AI search visibility, three things decide a citation, and llms.txt is not one of them. First, make sure the search crawlers can reach you at all - the per-engine list is in how to allow AI crawlers. Second, publish pages an engine can lift a clean answer from, which is the craft of answer engine optimization. Third, earn mentions from sources the engines already trust, which is the heart of AI search visibility. Do those and you are working the levers that actually move citations; an llms.txt file ranks below all three and substitutes for none of them.

If you want to, go ahead - it is a static file and it costs little. Just rank it honestly: last, after the three things above, and with no expectation of a citation lift. There is one rule if you do publish it. Keep its content faithful to your real pages. Putting a different, optimized story in llms.txt than what users and crawlers see is cloaking - the same unverifiable, owner-controlled signaling that gets the format dismissed rather than trusted. An honest llms.txt is harmless; a deceptive one is a liability for no payoff. And if you are unsure whether yours is doing anything, the only way to know is to measure your citation share with and without it, which is the subject of measuring AI citations.

It is easy to lose a month on tactics that feel productive and move nothing. The fastest way to skip that is to see where you actually stand: our free 10-minute AI search audit checks the things that decide citations, and the Free Snapshot reads what the engines say about your category today. If the answer is that you are missing, the SXO Audit runs 25 buyer prompts across six engines with three trials each over time, so you spend your effort on the levers that move your citation share rather than on a file the engines are not reading.

Try the Free Snapshot   See the SXO Audit

By the Web Cited Editorial Research Team. Last updated 1 June 2026.