llms.txt is the most argued-about file in AI search, and most of the arguing is between people selling it as a ranking factor and people dismissing it because it is not one.
Both are missing what it actually does.
llms.txt is a proposed standard for a markdown file at the root of your site that gives AI agents a concise, expert-level summary of what the site is and where its important pages are. That is the whole idea. What follows is where it came from, what has actually happened to it, and whether it is worth twenty minutes of your afternoon.
Where it came from
Jeremy Howard of Answer.AI proposed it on September 3, 2024. The reasoning is plain enough that it has survived two years without needing a rewrite.
Web pages are built for people. An HTML page wraps its information in navigation, ads and JavaScript, and converting that back into clean text is both difficult and lossy. Context windows, while larger than they were, are still too small for most sites in their entirety, and every wasted token costs time and money. So: put a concise version somewhere predictable.
The proposal reached a second version in 2026, revised on the strength of two years of adoption rather than theory.
What has actually happened since
This is where most writing on the subject is out of date, so here is the current state as the proposal itself describes it.
Thousands of sites now publish an llms.txt file. Documentation platforms generate them automatically. The AI labs publish them for their own developer documentation, including OpenAI, Anthropic and Google.
And the detail that matters most: Chrome's Lighthouse now audits sites for one as part of its agentic browsing checks. When a mainstream developer tool starts checking for a convention, the convention has stopped being a proposal and started being an expectation.
None of that is the same as evidence that it changes answers.
For the wider discipline this sits inside, see what generative engine optimization is.
Does it work?
Depends entirely on what you mean.
If you mean "will it get me cited by ChatGPT," there is no evidence that it will. No major assistant has publicly confirmed that reading an llms.txt file changes whether a brand gets named in an answer. Anyone selling it to you on that basis is overreaching, and you should ask them for the measurement.
Being cited depends on retrieval, and retrieval depends on the things it has always depended on: being crawlable, being relevant to the question, and answering that question in prose a model can lift. Google is explicit that its generative features are rooted in core Search ranking and quality systems rather than running on a separate signal, and that there is no special markup you need to add for generative AI search. An llms.txt file is not an exception hiding in that guidance.
If you mean "does it do something useful," yes, and the useful thing is underrated. Any agent that fetches it gets your description of your own product, in clean form, in your words. That matters because the alternative is worse. In most categories the pages describing you are mostly not yours, which means roundups, comparison posts and forum threads are briefing the model before your site does. An llms.txt file is one of the very few places you control that text exactly.
If the answer you want is whether any of this changed your visibility, that is a measurement question rather than a file question.
llms.txt is not robots.txt
The similar name causes genuine confusion, so it is worth being blunt.
robots.txt controls access. It tells crawlers what they may and may not fetch, and the assistants use named crawlers you can allow or block: Google publishes its list including Google-Extended, OpenAI documents its crawlers separately by purpose, and Perplexity documents PerplexityBot.
llms.txt controls nothing. It is a briefing document for a reader that has already arrived.
One is a gate. The other is a handout. If your robots.txt blocks the crawler, your llms.txt will never be read, which is the order most audits get backwards.
What to actually put in it
The format is a title, a blockquote summary, then sections of links with one-line descriptions. That is it. The interesting decisions are about content, not syntax.
Write sentences a model can quote and be correct. The test is whether any single line, lifted alone, still says something true and specific about you. "Industry-leading AI visibility platform" fails that test. "Tracks six assistants, reports each separately, never blends them into one score" passes.
Put your prices in it. Pricing is the fact assistants most often get wrong about a brand, usually because they are working from a comparison post written eighteen months ago. Your own file is the cheapest correction available.
Put your constraints in it. This is the counterintuitive one. List what you do not do. We say plainly that we do not track Grok, Llama or DeepSeek, have no REST API, and offer no historical backfill. A model that repeats those limitations is describing us accurately, and accurate beats flattering when the alternative is a trial user discovering it on day two.
Generate it, do not hand-write it. Ours is rendered at build time from the same source files the pricing page reads. A hand-maintained file is one price change away from lying, and a file that lies to a model is worse than no file. We learned that from the version of ours that quietly advertised a plan ladder we had retired.
What we do
You can read ours. It states what the product is in a form designed to be quoted, lists the plans with their real prices, names the six engines and explicitly names the three we do not track.
We also publish a markdown twin of every blog post at the same URL with a .md extension, linked from the page, so an agent can fetch clean text without parsing our HTML at all.
None of this is a growth tactic. We sell AI visibility measurement, so a site that assistants cannot read cleanly would be an argument against the product. The file is the dogfood.
The honest recommendation
Add one. It takes twenty minutes, it costs nothing, and it puts your own description of yourself where an agent can find it.
Do not expect it to move a number, and do not let anyone charge you for it as an AI search service. If you want to know whether anything you do moves the number, that is a measurement question, and the answer involves running a fixed prompt set on a schedule and reporting each engine separately rather than trusting a file to do the work.
For the discipline this sits inside, see what generative engine optimization actually is. For the definition underneath all of it, see what AI visibility is. And the plans are here if you want to find out whether any of it changed anything.