How to create an llms.txt file
Two routes: generate a draft in about a minute and edit it, or write it by hand in twenty. Either way, the work that matters is choosing pages and writing descriptions.
Last updated
To create an llms.txt file, list your most useful pages in Markdown under a few H2 headings, give each a one-line description, and publish the file at /llms.txt on your domain. The seven steps below take a typical site from nothing to a validated, live file in under an hour.
Step 1: what should the file help a model answer?
Before listing anything, finish this sentence: when a model reads this, I want it to be able to answer…. For a shop that is what you sell and what it costs. For a software product it is what the thing does, how to integrate it and what it costs. For a consultancy it is what you do and who for.
That sentence decides which pages belong. Everything that helps answer it goes in. Everything else is left out, or listed in a final ## Optional section if it is useful background.
Step 2: which pages should you include?
Aim for somewhere between fifteen and a hundred URLs. The instinct is to include everything, and it is the wrong instinct: a file where every page is important tells a model nothing about which page is important.
| Include | Leave out |
|---|---|
| Product and service pages | Tag, category and date archives |
| Documentation and API reference | Paginated listings (page 2, 3, 4…) |
| Pricing | Login, cart, checkout and account pages |
| Substantial guides and explainers | Thin or duplicate content |
| Support and FAQ pages | Anything noindex or behind a login |
| A single good About page | Press releases from 2019 |
Step 3: generate a draft
Paste your sitemap or site address into the generator. It fetches each page, reads the title and meta description, groups the results by URL path and returns a formatted file. It leaves out archives, feeds, search and account pages for you. That gets you most of the structure and a first pass at every description.
If you would rather write it from scratch, copy the skeleton from the format guide, or start from the example in the llms.txt proposal at llmstxt.org.
Step 4: rewrite the descriptions
This is the step that makes the difference, and the one people skip. Generated descriptions come from your meta tags, which were written for search snippets, not for a model deciding whether to open a page. They are often vague or repeated across a whole section.
Rewrite them to say what is actually on the page and, where it matters, what is not:
Generated from the meta description: - [Pricing](https://example.com/pricing): Simple, transparent pricing for teams of every size. Rewritten to be useful: - [Pricing](https://example.com/pricing): Three plans from USD 12 to USD 79 per user per month. Annual billing, no free tier, 14-day trial.
The second version lets a model answer a pricing question correctly without fetching anything. The first tells it nothing at all. Aim for one line, under about twenty-five words, specific enough to be wrong.
Step 5: where do you publish it?
Put it at https://yourdomain.com/llms.txt, served as plain text. The root is the place every reader checks. Since V2 (see the V2 change notes), a file in a subdirectory such as /docs/llms.txt is also valid and covers the pages beneath it, which helps when a separate team runs that section; keep one at the root as well. Platform instructions: WordPress, Shopify, Next.js.
Step 6: how do you check it works?
Two checks, both quick. Open the URL in a private browser window and confirm you see raw Markdown rather than a rendered page, a redirect or a 404. Then run it through the validator, which catches structural mistakes and checks how the file is served.
The most common failure is a platform serving the file as HTML. It looks right in a browser because the text is all there, but the content type is text/html and parsers reject it. The validator checks this for you.
Step 7: keep it current
Review it every quarter, and regenerate it whenever you launch a product area, restructure your URLs or change your pricing. A file describing last year’s product line is worse than no file, because it will be believed.
Frequently asked questions
How long should this take?
Twenty minutes to an hour for a typical site. Generating the draft takes a minute; the rest is choosing pages and rewriting descriptions, which is the part worth your time.
Can I automate it in my build?
Yes, and for documentation sites you should. Generate the file at build time from your page metadata so it cannot drift out of date. The Next.js guide shows a route-handler approach.
Do I need llms-full.txt as well?
Only if you have substantial documentation worth quoting word for word. Publish llms.txt first; it is the more useful file for almost every site.
What if my site has thousands of pages?
List the sections rather than the pages. Link to your documentation index rather than four hundred reference pages, and describe what the index contains.