llms.txt news
What has actually happened with llms.txt since it was proposed in 2024. Every post is dated, explained in plain words and linked to its original source, so you can check it yourself.
Last updated
Where do things stand?
llms.txt is a widely published, lightly used convention. In short, as of the date at the top of this page:
| Question | Answer | Read more |
|---|---|---|
| Is there a standard? | A community proposal, revised as V2 on 10 August 2026. No standards body is behind it. | V2 is published |
| Does Google use it? | Google Search ignores it, and says publishing one is fine. Lighthouse checks it, experimentally. | Google’s position |
| Do AI assistants read it? | None of Google, OpenAI, Anthropic or Perplexity says in its documentation that its crawlers read it. Coding agents do look for it. | Agents and llms.txt |
| Is anyone reading the files? | Rarely: 97% of the files Ahrefs could see got no requests in May 2026. | The Ahrefs study |
| Is there a risk? | Coding agents have run install commands found in llms.txt files. List only names you own. | The security research |
2026
-
Common Crawl looks inside 584,107 llms.txt files
Common Crawl analysed every non-empty llms.txt in its July 2026 crawl. Most were made by plugins or templates, nearly a quarter had no links, and some tried to give crawlers orders.
-
Coding agents ran install commands they found in llms.txt files
A researcher registered package names that companies’ llms.txt files told agents to install. Within minutes, an AI coding agent inside a Fortune 500 company had run one.
-
llms.txt V2 is published
Two years after proposing it, Jeremy Howard revised llms.txt. Existing files stay valid; the big additions are subdirectory files and links to Markdown versions of pages.
-
Google: Search ignores llms.txt, and publishing one is fine
A month after publishing its generative AI guide, Google clarified the llms.txt line: Search ignores the file, so it cannot help or hurt your rankings, and publishing one is fine.
-
Ahrefs: 97% of llms.txt files got no requests in a month
Ahrefs looked at who actually requests llms.txt. Across 38,360 sites with a valid file, 97% received no requests at all in May 2026, from bots or people.
-
Google publishes its guide to optimizing for generative AI
Google’s new guide said you do not need AI text files to appear in its generative AI features, and listed creating llms.txt among the things to skip.
-
Lighthouse adds an llms.txt check to its agentic browsing audits
Google’s Lighthouse tool now checks llms.txt in an experimental “agentic browsing” category. It looks for an H1, a link and some content, nothing more.
2025
-
An llms.txt appears on Google’s Search docs, then disappears
An llms.txt turned up on Google Search’s developer documentation, apparently added by Google’s documentation system, and was soon removed. It was not an endorsement.
-
SE Ranking finds no link between llms.txt and AI citations
SE Ranking compared nearly 300,000 domains with how often AI answers cited them. Having an llms.txt made no difference, and leaving it out of the model made the model more accurate.
-
John Mueller: no AI system currently uses llms.txt
Replying to a question on Bluesky, Google’s John Mueller wrote: “FWIW no AI system currently uses llms.txt.”
-
John Mueller compares llms.txt to the keywords meta tag
On Reddit, Google’s John Mueller said llms.txt is “comparable to the keywords meta tag”: a site owner’s own claim about what the site is about.
2024
-
Jeremy Howard proposes llms.txt
Writing for Answer.AI, Jeremy Howard proposed a Markdown file at the root of a site to tell language models what the site is and where its most useful pages are.
How these posts are written. Every post links to its original source: the proposal itself, the company’s own documentation, the person’s own post or the researchers’ own write-up. Coverage links are extras. Press releases, product launches and claims we cannot trace to a source are left out, and if a post turns out to be wrong it is corrected in place, with a note.
Who publishes an llms.txt?
Developer documentation is where the file has caught on. Each of these was fetched from the company’s own site on 17 October 2026 and returned a real llms.txt:
| Company | File | Notes |
|---|---|---|
| Anthropic | platform.claude.com/llms.txt | API documentation; also links an llms-full.txt |
| OpenAI | developers.openai.com/llms.txt | Developer documentation |
| Perplexity | docs.perplexity.ai/llms.txt | API documentation |
| Stripe | docs.stripe.com/llms.txt | Documentation |
| Cloudflare | developers.cloudflare.com/llms.txt | Developer docs, with one file per product |
| Vercel | vercel.com/llms.txt | Documentation; also links an llms-full.txt |
| Mintlify | www.mintlify.com/docs/llms.txt | Generates one automatically for every docs site it hosts |
| GitBook | gitbook.com/docs/llms.txt | Publishes one automatically for docs sites |
Publishing a file is not the same as reading one. OpenAI, Anthropic and Perplexity each publish an llms.txt for their own docs, but none of them says in its crawler documentation that it reads other sites’ files.
Frequently asked questions
Does Google use llms.txt?
Not for Search. Google says Search ignores AI text files such as llms.txt, so they neither help nor hurt rankings, and that publishing one is fine. Its Lighthouse developer tool does check the file, in an experimental audit, which is a quality check rather than a ranking signal.
Do ChatGPT, Claude or Perplexity read llms.txt?
Their makers do not say so. OpenAI, Anthropic and Perplexity each publish an llms.txt for their own developer docs, but their crawler documentation covers robots.txt only. Coding agents are a different case: research in August 2026 showed them looking for llms.txt on their own while doing integration tasks.
Is it safe to publish an llms.txt?
Yes, if it is accurate. Treat it as instructions an agent might follow: list only packages, commands and domains you own and still control, and remove examples you no longer maintain. Never put access rules in it; those belong in robots.txt.
How often are posts added?
Whenever something with a primary source happens. Posts are also listed in the site’s RSS feed.