llms.txt is a plain text file, written in Markdown, that sits at the root of your website (yoursite.com/llms.txt) and gives AI models such as ChatGPT and Claude a curated map of your most useful content. It was proposed by developer Jeremy Howard of Answer.AI in September 2024, and the full specification lives at llmstxt.org. Think of it as a sitemap, but instead of listing every page for Google, it points AI systems to the handful of pages you actually want them reading and quoting. Whether you need one depends on how much of your enquiries already come from AI search tools, and that is worth checking before you spend a day building a file nobody reads.
I first came across llms.txt while going through a client’s site structure for a project involving their SEO services. Their developer had already added one out of curiosity, and it took him roughly 40 minutes. That is the appeal. It is cheap to build, it does no harm, and if AI search genuinely grows the way most of the industry expects, having one in place now costs you very little.
How llms.txt Actually Works
The format is deliberately simple. You start with an H1 with your site or brand name, then a one line blockquote summary, then H2 sections such as Docs, Products or Blog, each containing a short list of links with a one line description. There is no strict schema to follow beyond that, which is part of why it caught on quickly among developer tools and SaaS platforms. If you have read our piece on technical SEO fundamentals, the logic will feel familiar. It is the same principle as a clean sitemap, just aimed at a different reader.
llms.txt vs robots.txt vs Sitemap.xml
People often confuse the three, so here is how they differ.
| File | Purpose | Audience |
| robots.txt | Tells crawlers what they may or may not access | Search engine and AI bots |
| sitemap.xml | Lists every indexable URL on the site | Search engines |
| llms.txt | Curated, editorial list of your best content | Large language models |
None of these files replace the others. If your structured data and schema markup is already solid, llms.txt is simply another layer on top, not a substitute for it.
Do You Actually Need One
Here is the honest bit. No major AI provider has formally confirmed that its crawler reads llms.txt as part of ranking or citation logic. Anthropic publishes its own file for developer docs, and tools like Mintlify and Cloudflare use it, but adoption among the big players such as OpenAI and Google remains unconfirmed rather than guaranteed. Ahrefs covered this gap well in its own analysis, and it is worth a read if you want the sceptic’s view.
That said, if you run a content heavy site, publish through a PPC marketing agency approach alongside organic, or care about how you show up when someone asks ChatGPT a question about your industry, the downside of adding llms.txt is close to zero. We touched on this shift in our piece on AI enhanced search, and the direction of travel has not changed since.
How to Create One
Building your own does not need a developer, though having one helps.
- Start with your homepage summary in one sentence
- List your most important pages, not every page, under clear H2 headings
- Keep descriptions short and factual, no marketing fluff
- Upload the file to your root directory as /llms.txt
- Review it every quarter as your content changes
If you also run Google Ads campaigns or invest in Google Ads training, it is worth cross checking that your llms.txt reflects the same priority pages your paid strategy sends traffic to. We covered a similar audit process in our guide to Google Search Console, which pairs well with this exercise. And if you are unsure which pages matter most, our thoughts on where SearchGPT optimisation is heading might help you prioritise.
FAQs
Does llms.txt affect my Google rankings?
No. It has no confirmed relationship with traditional Google search rankings. It is aimed at how AI models read and cite your content.
Is llms.txt mandatory?
No, it is a voluntary proposal, not an official web standard, so there is no requirement to add one.
Can I use llms.txt instead of a sitemap?
No. They serve different audiences and should sit alongside each other, not replace one another.
If you would like help auditing your site for AI visibility alongside your existing social media advertising or SEO work, get in touch with Flow20 and we will talk you through what is actually worth prioritising.
