Was ist eine llms.txt?

Websites are no longer visited only by humans, but increasingly read and evaluated by AI systems and AI agents too.

Pages designed for humans, however, contain many elements that aren't particularly helpful for an AI system: navigation, footers, ads, scripts, and layout markup. All of that costs context without providing any additional content value.

This is exactly where llms.txt comes in.

Where Does the Idea of llms.txt Come From?

llms.txt was proposed by Jeremy Howard in September 2024 and published at llmstxt.org.

The basic idea: alongside its normal HTML pages, a website provides a simple markdown file at /llms.txt. This file contains a compact overview of the most important content, prepared specifically for AI systems – with short descriptions and links to the relevant pages.

How Is an llms.txt Structured?

The structure of an llms.txt follows a simple, clearly defined format:

• an H1 heading with the name of the project or company (the only required part),
• a short introductory blockquote (>) with a brief summary,
• optionally, further explanatory paragraphs,
• any number of H2 sections with link lists and short descriptions,
• optionally, a closing ## Optional section for supplementary, less central content.

A simple example:

A system evaluating a website for AI purposes thereby finds a compact, clearly structured overview – without having to fully crawl or interpret the actual website.

What Problem Does llms.txt Solve?

AI systems work with a limited context window. Every piece of information that gets processed costs space – including navigation bars, cookie notices, or ad space that humans naturally see as just part of the website.

An llms.txt, by contrast, provides a curated, editorially prepared overview from the outset.

Instead of crawling an entire website page by page and filtering the relevant content out of the HTML code, a system can access a compact file prepared specifically for exactly this purpose directly.

Is llms.txt an Official Web Standard?

No – llms.txt isn't an official standard from an institution like the IETF or the W3C; it started out as a proposed convention.

Even so, the file has become fairly widely established within a short time. Numerous developer tools now automatically look for an llms.txt or llms-full.txt as soon as they're pointed at a documentation website – including Cursor, Windsurf, Claude Code, GitHub Copilot, Cline, and Aider, among others.

In June 2026, there was also a proposal to officially standardize llms.txt through the W3C. Whether, and in what form, that will actually turn into a recognized web standard remains an open question at this point.

Version 2 of llms.txt (Since August 2026)

In August 2026, the llms.txt specification was revised – nearly two years after its original publication.

The actual file format itself hasn't changed: an existing llms.txt built to the original version remains valid under version 2 as well.

What has changed, above all, is how an llms.txt gets discovered. Version 2 adds so-called link relations for this:

• rel="alternate" can point from an individual page to its own markdown version,
• rel="describedby" can point from a page to the llms.txt that covers it.

These can be delivered either as an HTML <link> element or as an HTTP Link header. This means AI systems and other tools no longer need to guess or simply try /llms.txt by default – they can locate the right file more precisely instead.

What llms.txt Doesn't Replace

An llms.txt isn't a confirmed Google ranking factor and shouldn't be used with that expectation.

It also doesn't replace any of a website's actual technical foundations: clean semantic HTML, an understandable accessibility tree, structured data based on Schema.org, and good information architecture remain the real basis for a website being understandable – by humans and machines alike.

That tools like Chrome Lighthouse are increasingly addressing how well a website is accessible to agents, though, shows that this topic isn't a short-lived trend.

An llms.txt can be a useful additional layer in this context – a compact, purpose-built entry point for AI systems, not a replacement for a technically well-built website.

Conclusion

llms.txt is a simple, standardized markdown file that lets websites make their most important content available specifically for AI systems and AI agents.

It's not an official web standard and not a ranking factor, but it has established itself as a practical convention within a short time – supported by numerous AI-powered developer tools and now even the subject of a standardization proposal at the W3C.

How an llms.txt can be generated automatically from the page tree for TYPO3 websites, instead of having to be maintained by hand, is something we describe in our article about our own TYPO3 extension llms_txt.