llms.txt not working? Find the cause
Almost every broken llms.txt is one of five things: the file is in the wrong place, has the wrong name, comes back as a web page, sits behind a redirect, or is an old copy from a cache. Each takes a minute to check.
Last updated
Start by asking your own site for the file, exactly as a tool would. Open https://yourdomain.com/llms.txt in a private window, then run the same address through the llms.txt validator. The validator fetches the live file and reports how it was served, not only what it says.
If you prefer a terminal, one command shows the three things that matter: the status code, the content type and any redirect.
What a healthy response looks like
curl -sI https://yourdomain.com/llms.txt HTTP/2 200 content-type: text/plain; charset=utf-8 cache-control: public, max-age=3600
Which symptom do you see?
| What you see | Likely cause | Fix |
|---|---|---|
| 404 Not Found | The file is not in the folder your domain serves, or it is named llm.txt or llms.text. | Put it in the web root (often public_html/, or public/ in a framework) and name it exactly llms.txt. |
| 200, but it shows your home page or a styled page | A single-page app or CMS answers every unknown address with HTML. The file never made it into the build. | Add the file where the build copies static files, and make sure the catch-all route leaves /llms.txt alone. |
| 301 or 302 first, then the file | www versus no www, http to https, or a platform workaround such as Shopify’s redirect. | Link and test the final address. Some readers do not follow redirects, so serve the file directly where you can. |
| You edited the file, but the old one still loads | A CDN or browser cache is serving its copy until it expires. | Purge the cache in your host or CDN panel, or wait for the time in cache-control. |
| It loads, but no AI tool seems to use it | Nothing is wrong with the file. Reading it is up to each tool. | See the last section: a valid file is all you control. |
Why does the file come back as HTML?
This is the most common failure on modern sites. Frameworks and site builders often send every address they do not recognise to the same page, so the browser shows your home page and the status is still 200. A tool asking for /llms.txt then receives HTML, not Markdown. Our validator treats a text/html response as an error for that reason, and asks for text/plain or text/markdown. The platform guides for Next.js, WordPress and Shopify show where the file has to go on each.
Why do I still see the old version?
Most hosts put a cache in front of static files. Our own llms.txt is a good example: on 22 October 2026 it was served with cache-control: public, max-age=3600, and the copy our check received was already 40 minutes old. After an edit, a cached copy like that can take up to an hour to change. The age header in the response tells you how old the copy you received is.
Check the live file after every change. Run the validator against the address, not against the file on your computer. That is the only way to see what readers actually receive.
What does the validator check when you give it a URL?
On top of the structure checks it runs on any file, the validator reports how the file was served:
| Check | How it is reported |
|---|---|
| The address could not be fetched (for example HTTP 404) | Shown instead of a report, with the status |
Served as text/html | Error |
| Served as another type that is not plain text or Markdown | Warning |
| The address redirected before the file | Warning |
The file has a name other than llms.txt or llms-full.txt | Warning |
| The file sits in a subfolder | Note: under V2 a file in a subfolder covers the pages under it (the V2 change notes) |
The file is fine. Why is nobody reading it?
Because reading it is optional. The file follows the llms.txt proposal at llmstxt.org, which each AI company decides whether to use. Google says Search ignores llms.txt (Google’s guide to optimizing for generative AI), and an Ahrefs study found that most files received no requests in a month. A correct, reachable file is the part you control. Whether a given tool reads it is not.
Frequently asked questions
What content type should llms.txt have?
Plain text (text/plain) or Markdown (text/markdown). The file is Markdown, so an HTML response means something other than the file was returned.
Can llms.txt live in a subfolder?
Yes, since V2: a file at /docs/llms.txt covers the pages under /docs/ (the V2 change notes). Tools written before V2 only look at the root, so keep one at /llms.txt as well.
How long until a change shows?
As long as your cache keeps the old copy. Look at cache-control and age in the response, or purge the cache in your host or CDN panel.
Do I need llms-full.txt too?
No. It is optional. llms.txt vs llms-full.txt explains when it helps.