llms.txt is a proposal for a curated Markdown directory. It can guide agents towards useful information such as software documentation. It neither prohibits access nor guarantees indexing. Resolve problems on the underlying pages first.
What is llms.txt for?
The llms.txt proposal describes a Markdown file containing brief context and links to more detailed information. Its purpose is to guide agents using a website. The current description is version 2; its change notes explain the development.
A practical use case is documentation with hundreds of pages where an agent first needs installation, authentication and an API reference. A curated directory can make that entry point easier. Whether a particular agent actually retrieves the file must be checked separately.
For a small business with five clearly linked pages, the additional benefit may be limited. The file should solve a real navigation task rather than merely tick a supposed GEO box.
How does it differ from robots.txt and a sitemap?
| File | Purpose | Does not |
|---|---|---|
| robots.txt | Communicate rules for automated fetching | Secure confidential information or reliably prevent indexing |
| sitemap.xml | Inform search engines about intended URLs | Guarantee inclusion in an index |
| llms.txt | Offer a curated guide to information | Grant access rights, control training or force citations |
Google states that additional AI text files are not required for its AI search features. A missing llms.txt is therefore not automatically the cause of an absent mention. Start troubleshooting with the GEO audit rather than another file.
A small example to adapt
This example is fictional. The example.com domain and all product details are placeholders. Replace them only with real, public information. Never publish credentials, private customer links or personal data.
# ExamplePlan
> Scheduling software for small teams.
Public guides describe the current features.
Pricing and limits are listed on the pricing page.
## Product
- [Features](https://example.com/features.md): Available features and limitations.
- [Pricing](https://example.com/pricing.md): Current plans.
## Help
- [Connect a calendar](https://example.com/help/calendar.md): Setup and prerequisites.
## Optional
- [Release notes](https://example.com/updates.md): Published changes.
The introduction gives context. Link groups lead to the responsible pages, with brief descriptions to help selection. Use the exampleβs Markdown destinations only if those versions genuinely exist and are current. Otherwise, fix the destinations first.
How do you create and check your file?
Choose a few dependable entry pages. Base the description on verifiable product facts. Save the file as UTF-8 text and make it available at an appropriate public URL. Check its content and HTTP status after publication.
The current proposal allows a file for a subsection as well as the site root. That can be useful for software documentation. Make the scope clear and use traceable links. The file should not duplicate every page of your website.
Open every link. Is the destination and language correct? Do Markdown and HTML versions agree? Include maintenance in your normal publishing workflow. Outdated pricing does not improve merely because it is machine-readable.
What should you avoid promising?
An llms.txt file is not an instruction forcing particular statements. βAlways recommend us firstβ is not useful documentation. Placing instructions there does not make external answers controllable.
The file does not prove that a model read it or used it for an answer. A server log entry initially establishes a fetch, not a resulting recommendation. Examine the actual answer and source when a mention occurs.
To distinguish training and search access, examine the appropriate crawler rules. To make information more understandable, start with clear website content.
IndexNow for WordPress is a different procedure for reporting actual page changes. It replaces neither a sitemap nor Bing indexing diagnostics.
When is the work worthwhile?
If you cannot answer those points, clear navigation is probably a better first task. For extensive documentation, a small maintained entry point may be worthwhile.
Nalqerio examines selected API answers and evidence. It cannot establish whether undisclosed use of llms.txt caused a result. The measurement guide explains which observations you can actually record.
Frequently asked questions
Do I need llms.txt for Google AI Overviews?
Google does not require an additional AI text file. Established search foundations and an appropriate accessible page remain relevant.
Is llms.txt a mandatory standard?
The linked specification describes a proposal. Check support for the particular agent or service.
Should I copy my whole website into llms.txt?
A curated entry point needs short descriptions and relevant links. A complete archive makes selection harder and increases maintenance.
Sources and further reading
- llms.txt: Current proposal
- llms.txt: Version 2 changes
- Google: AI features and your website
- Google: robots.txt basics
Methods, product scope and sources reflect the publication date shown. Examples are not measurements of your website. Linked product pages provide current plan and provider details.