As AI assistants and AI-powered search become more common, website owners are looking for ways to make their content easier for large language models to understand. One emerging option is llms.txt, a lightweight file that highlights a website’s most useful resources.
It is not a guaranteed ranking tool and does not replace search engine optimisation. Instead, it acts as a curated guide that may help compatible AI tools understand what a website covers and where its important information can be found.
What Is llms.txt?
An llms.txt file is a Markdown-formatted document usually placed in a website’s root directory, such as example.com/llms.txt. Jeremy Howard introduced the proposal in September 2024 to help large language models access concise website information at inference time.
Modern websites often contain navigation menus, scripts, repeated layout elements and many pages. These features serve human visitors but can make it harder for an AI system to identify the most relevant content quickly. llms.txt provides a simplified summary and selected links to important resources.
A company could use it to highlight its services, company profile, documentation, policies, FAQs and useful articles.
How Does llms.txt Work?
The file uses Markdown, making it readable by humans and machines. Under the proposed format, the only required element is an H1 heading containing the website or project name. It may also include a short blockquote, explanatory notes and H2 sections containing links with brief descriptions.
A simple example could look like this:
# Example Company
> Example Company provides digital solutions.
## Services
- [Web Design](https://example.com/web-design): Custom website services.
- [Digital Marketing](https://example.com/marketing): SEO and online marketing.
## Resources
- [Blog](https://example.com/blog): Guides and industry insights.
A compatible AI agent can use these links as starting points instead of interpreting the whole website immediately.
llms.txt vs robots.txt and XML Sitemaps
These files have different purposes. A robots.txt file provides crawling instructions, while an XML sitemap lists indexable URLs for search-engine discovery. llms.txt provides context and directs language models towards a curated selection of useful content. It is designed to coexist with existing web standards rather than replace them.
Businesses still need a technically sound and well-organised professional website design. Clear navigation, useful content, internal links, structured data and crawlable pages remain important.
What Should You Include?
Focus on pages that accurately explain your organisation and answer important user questions. These may include core service or product pages, company information, documentation, support resources, policies, FAQs and selected case studies.
Avoid copying every URL from your sitemap. The value of llms.txt comes from careful selection, logical grouping and clear descriptions.
For a company investing in a broader digital marketing strategy, the file could group links under services, expertise, resources and company information. Each description should explain what the linked page contains.
How to Create an llms.txt File
First, identify the pages that best represent your business. Create a plain-text file, add an H1 title, include a concise summary and organise important links under descriptive H2 headings.
Save the document as llms.txt, upload it to the root directory and confirm that it is publicly available at https://yourdomain.com/llms.txt. Chrome’s Lighthouse documentation recommends placing the file at the root and following the proposed Markdown format.
Review the file whenever important URLs, services or documentation change. Regular website maintenance helps prevent it from linking to outdated or removed content.
Should Your Website Use llms.txt?
Adding the file may be worthwhile because it is simple to implement and gives compatible AI agents a cleaner overview of a website. It may be especially useful for documentation websites, software businesses, universities, e-commerce stores and content-rich organisations.
However, expectations should remain realistic. llms.txt is still an emerging and optional convention. Chrome Lighthouse currently marks the audit as not applicable when the file is missing, showing that websites are not required to provide one.
There is no guarantee that it will improve Google rankings, generate AI citations or be used by every AI platform. Its value depends on platform adoption and the quality of the pages included.
Final Thoughts
llms.txt is a practical way to present important website content in a concise, AI-friendly format. It works as a curated guide, not a replacement for robots.txt, XML sitemaps or traditional SEO.
For businesses preparing for AI-driven discovery, creating the file can be a sensible forward-looking step. It should support a wider strategy built around trustworthy content, strong website architecture, technical SEO and regular updates.