llms.txt is a proposed web standard: a plain Markdown file, placed at your domain root, that gives large language models a clean, curated map of your most important pages. It matters because AI systems work from a limited context window, and a good llms.txt points them to the content you want cited, in a format they can read without parsing your full HTML. It is voluntary: no major AI provider has committed to reading it, so treat it as a low-cost signal, not a ranking lever.
You have spent years making your site readable for Google. AI assistants read it differently. When ChatGPT or Perplexity answers a question about your category, it does not crawl your whole site. It pulls a few pages into a small working memory and reasons from those.
If the model cannot tell which of your pages matter, it guesses, and it often guesses badly: a stale landing page, a tag archive, a page you would never choose. llms.txt is one attempt to fix that. It is a short file that hands AI systems a curated list of your best pages, in plain text they can read instantly.
This guide explains what llms.txt is, what goes inside it, how to create and place one in about an hour, and whether it is actually worth your time. It is written for marketers and site owners, not engineers.
What is llms.txt, and what problem does it solve?
llms.txt is a proposed standard, published at llmstxt.org and introduced in September 2024 by Jeremy Howard of Answer.AI. It is a single Markdown file that lists your most important pages, each with a short description, so an AI system can find them without wading through your site.
It helps to place it next to the two files you already know. robots.txt tells crawlers where they may not go. sitemap.xml lists every URL you have. llms.txt does something different: it curates the handful of pages you most want an AI to read, in human-readable text.
The problem it targets is the context window. A language model can only hold so much at once. It cannot load your entire site, and much of your HTML is navigation, scripts, and markup it does not need. A clean index points it straight at your best content, in the format it finds easiest to parse.
What goes inside an llms.txt file (the format)
The format is CommonMark Markdown with a fixed shape, defined in the spec: one H1 with your site or project name, an optional blockquote summary, then H2 sections (for example "Learn", "Products", "Guides"), each holding a bulleted list of links with one-line descriptions. Every link should be an absolute https:// URL that returns a 200 response.
A short example looks like this:
# Citable
> Advisory for organic growth and AI-search visibility.
## Learn
- [What is AEO vs GEO vs SEO](https://citable.in/learn/what-is-aeo-vs-geo-vs-seo): plain definitions of the three approaches.
- [robots.txt for AI crawlers](https://citable.in/learn/robots-txt-for-ai-crawlers): how to allow or block AI bots.
## Services
- [Brand Visibility Audit](https://citable.in/services): a scored audit of a brand's search and AI presence.
You may also see llms-full.txt. Per GitBook's guide, that is a separate, optional file containing the full text of your key pages in one document, where llms.txt is only the curated index. Use llms-full.txt only if your content is documentation-heavy and you want the whole thing available in one fetch.
How to create and place your llms.txt
Creating one is a short, five-step job:
- List your key pages. Choose the 10 to 20 pages you most want an AI to read and cite: your best guides, core product or service pages, and your about page.
- Write the Markdown. Follow the shape above, or generate a first draft (many documentation platforms and free tools produce one). Keep each description to a single line.
- Save it as llms.txt and upload it to your domain root, so it resolves at
https://yourdomain.com/llms.txt. The spec puts the file at the root path. - Validate. Open the URL in a browser, confirm every link returns a 200, and avoid JavaScript-only pages: an agent that fetches the raw HTML of a script-rendered page gets nothing.
- Keep it current. Update the file whenever you publish, rename, or retire a page you want AI systems to know about.
Is llms.txt worth your time?
Here is the honest position. No major AI provider has publicly committed to reading llms.txt in production, and adoption is early. In a large study, Ahrefs found that about 28% of the domains it looked at published an llms.txt file, but 97% of those files received zero requests from AI agents.
So treat llms.txt as a low-cost, low-risk signal, not a growth lever. It takes about an hour, it cannot hurt your Google rankings, and it positions you cleanly if support grows. The sites that benefit most are documentation-heavy products, content publishers, and brands with a small set of high-value pages they want quoted accurately.
Frequently asked questions
Is llms.txt actually used by ChatGPT and Claude?
llms.txt is a voluntary standard that no major AI provider has confirmed reading in production. It costs little to publish and may help as support grows, but no engine currently guarantees it will use your llms.txt when answering.
What is the difference between llms.txt and llms-full.txt?
llms.txt is a short, curated index of links to your key pages. llms-full.txt is a separate, optional file containing the full text of those pages in one document. Most sites need only llms.txt; documentation-heavy sites sometimes add llms-full.txt.
Where exactly do I put the llms.txt file?
llms.txt belongs at the root of your domain, so it resolves at yourdomain.com/llms.txt, the same level as robots.txt. Placing it in a subfolder means most AI systems will not find it.
Does llms.txt help my Google ranking?
llms.txt does not affect Google Search rankings. It is intended for AI systems, not for Googlebot, and Google's ranking signals do not read it. Its only purpose is to make your key pages easier for language models to find.
How often should I update llms.txt?
llms.txt should be updated whenever you publish, rename, or retire a page you want AI systems to know about. For most sites that means a quick review every quarter, plus an edit each time an important page changes.
Where to go from here
llms.txt makes a small, honest promise: it makes your best pages easy for AI systems to find and read. It will not transform your visibility on its own, but it is an hour well spent and a clean foundation. The next question is whether AI engines can actually reach those pages, which is a matter of crawler access, and whether they are already citing you. Seeing that clearly is the work of a full Brand Visibility Audit.