llms.txt: what it is, why it matters and how to implement it

In December 2024, Jeremy Howard, the creator of fast.ai, proposed a simple standard: an llms.txt file placed in a domain's root directory, telling AI systems about the structure and content of the site. The standard is supported by Anthropic (Claude), Perplexity and a dozen or so other AI tools. Implementation cost: a few minutes. Potential effect: better understanding of your site by AI systems.

What is llms.txt?

llms.txt is a Markdown-format text file placed at yourdomain.com/llms.txt. Its job is to convey key information about the site to AI systems in a structured, easily processed form. The analogy: robots.txt tells search engines what they may index; llms.txt tells AI systems what is worth knowing about your site.

The standard is backed by platforms including Reddit, Medium, Cloudflare, Akamai and Creative Commons. It is not mandatory - AI systems manage without it. But its presence signals that your company understands AI mechanisms and wants to shape actively what AI knows about it.

What does an llms.txt file contain?

The file format is simple and human-readable. It consists of a few elements:

• A title and company description - one sentence explaining what the company does and for whom
• Links to key sections with descriptions - service pages, blog, about, case studies
• Optional key facts - the data you want AI to know about your company
• Optional guidance on content use - if you have preferences about citation

An example llms.txt for a B2B company

How to implement llms.txt, step by step

1. 1Create the llms.txt file - open a text editor and write the content in Markdown, as in the example above. Save it as llms.txt.
2. 2Place the file in the domain root - it has to be reachable at yourdomain.com/llms.txt. In Next.js put it in the /public folder. In WordPress, in the server root directory.
3. 3Check availability - open yourdomain.com/llms.txt in a browser. You should see the text file with its content.
4. 4Optionally add llms-full.txt - the standard also allows an extended version containing the full content of key pages, for models that want to fetch everything at once.

How to implement llms.txt in Next.js

In Next.js the simplest approach is to place the file in the /public folder, where it is served automatically as a static asset. Alternatively you can generate llms.txt dynamically through a route handler at app/llms.txt/route.ts, which lets you update the content without rebuilding the project.

Which AI systems read llms.txt?

Does llms.txt actually work?

The standard is too new for hard data on its effect on AI visibility. Early experiments suggest that AI systems officially supporting it - Claude and Perplexity - do use llms.txt to build context about a site. For models that do not support it formally, the file is simply ignored and does no harm.

Our conclusion: llms.txt is worth implementing on a cost-to-potential-benefit basis. Implementation takes 15 to 20 minutes. The risk is zero. The potential gain is better understanding of your site by the AI systems that support the standard. As the standard matures and adoption grows, companies that implemented it early will have nothing to catch up on.