llms.txt is a plain text file placed in a website's root directory. It is written in Markdown and tells AI models what the website is and which pages matter most. It is not meant for human readers, and it is not meant for Google's traditional search. Its audience is the models and AI agents behind tools such as ChatGPT, Claude and Perplexity.
The quickest way to understand it is to look at a real Hong Kong example. Open https://www.ud.hk/llms.txt and what you see is an llms.txt file. The first line is an H1 with the site name, followed by an introduction that explains what the company does and which language versions exist, and below that a list of links grouped into sections.
The format details and step-by-step writing instructions are already covered in full by the tool page article "llms.txt Guide: Help AI Understand Your Website". This article answers only three things: what its status is today, whether Hong Kong websites should add one, and how to check it once it is added.
How It Differs From robots.txt and sitemap.xml
Each of the three files handles one job. robots.txt decides who may come in, sitemap.xml lists what pages the site has, and llms.txt says which pages are the important ones. Search engines have been reading the first two for decades. The third is a new proposal that appeared only in 2024.
One point must be clear from the start: llms.txt cannot replace robots.txt. AI crawlers read robots.txt before entering a site, and if they are already blocked there, nobody will read your llms.txt however well it is written. For which crawlers are for training and which are for search, see the tool page article "AI Crawlers Explained". For the roles of OpenAI's three crawlers, see "What Happens If You Block GPTBot?" in this series.
The Honest Status: It Is a Proposal, Not a Standard
llms.txt was proposed by Jeremy Howard of Answer.AI on 3 September 2024. The specification page describes itself as "a proposal to standardise", which means it is a proposed specification, and it remains open to community changes today. The specification requires only one element: an H1 with the site name. The introduction, the sections and the link lists are all optional.
No major model provider has publicly committed to reading this file. Google's position is the clearest. The AI features optimisation guide from Google Search Central, updated in July 2026, states that websites do not need to create new machine-readable files, AI text files or Markdown in order to appear in Google Search, that Google Search itself does not use them, and that adding them neither helps nor harms rankings. Earlier, in April 2025, Google's John Mueller compared llms.txt to the keywords meta tag in a reply on Reddit, and said that no AI service had stated it was using the file at that time. The official crawler documentation from OpenAI, Anthropic and Perplexity offers website owners only robots.txt as a control method, and none of them says its crawlers read a site's llms.txt.
On the other side, Google's own Chrome team added llms.txt to the new "Agentic browsing" audit category in Lighthouse in May 2026 (the documentation marks it as experimental and it requires Chrome 150 or above). Lighthouse tries to read llms.txt from the site root. If the file is not found (404), the audit is only marked "not applicable". It flags a problem only when the request hits a server error, and the documentation recommends placing a file in the root directory that follows the llmstxt.org specification. In addition, the developer documentation sites of several model providers and infrastructure companies have an llms.txt themselves. In a test for this article on 17 October 2026, all of the URLs below returned 200:
| Website | URL | Content-Type |
|---|---|---|
| OpenAI developer documentation | developers.openai.com/llms.txt | text/plain |
| Anthropic documentation | docs.anthropic.com/llms.txt | text/plain |
| Cloudflare developer documentation | developers.cloudflare.com/llms.txt | text/plain |
| Chrome DevTools documentation | developer.chrome.com/docs/devtools/llms.txt | text/markdown |
Putting the two sides together, the conclusion is this: llms.txt has no effect on Google Search rankings, and there is no official commitment that it affects citations in answer engines such as ChatGPT. Its value lies in helping AI agents and reading tools grasp a site's structure faster, and Chrome's audit tooling has already started checking for it.
Should Hong Kong Websites Add One?
Add one: company websites with several service pages or product lines, SaaS websites with a documentation area or knowledge base, and bilingual Chinese and English websites. These three types of site have many pages and scattered priorities, so an AI agent needs more time to grasp the structure. An llms.txt can list the key points in thirty lines, and a bilingual site can also set out the matching URLs of both language versions in one place.
Can wait: single-page websites, restaurant or small shop websites with only a few pages, and websites whose content is almost never updated.
Do other things first: robots.txt blocks AI crawlers, the CDN or firewall in front of the site blocks AI crawlers as malicious traffic, or the pages have no structured data. These items matter far more than llms.txt and should be fixed first. For the order of work, refer to the earlier "GEO guide" on this site.
There is really only one test. The file costs half an hour to produce, and maintaining it means adding one line each time an important page is added. With a cost this low, as long as the site does not fall into the "can wait" group, adding one brings only benefits.
A Minimum Viable Version
There is no need to move the whole website into the file. One H1, one introduction, and three to five H2 sections with two or three links each are enough. SMEs can start with three sections: services, pricing and contact.
# Company name
> A two or three sentence introduction: what the company does in Hong Kong, which type of customer it serves, and that Chinese and English versions exist.
## Services
- [Service page title](https://www.example.com.hk/en/services): one sentence of description
## Pricing
- [Pricing page title](https://www.example.com.hk/en/pricing): one sentence of description
## Contact
- [Contact page title](https://www.example.com.hk/en/contact): opening hours and how to enquireThe sentence after each link should describe the actual content, for example "models and maintenance plans for commercial refrigerators". Do not write "learn more".
Three Common Mistakes
Wrong location. A typical website should place the file in the domain root, that is your-domain/llms.txt. The second version of the specification, from August 2026, allows the file to sit in a subpath such as /docs/llms.txt, covering the pages under that path. However, the Chrome Lighthouse audit recommends the root directory, and if the file exists only in a subpath and not in the root, the audit may treat the site as having none. Each subdomain is also independent: if the main site and the blog are on different subdomains, each needs its own file.
Served as HTML. Some content management systems wrap new pages in HTML. llms.txt must be plain text, and the Content-Type returned by the server should be text/plain (some sites use text/markdown, which is equally readable). If the server returns text/html, the model will read a pile of tags.
A long list of internal pages that mean nothing to customers. Privacy policies, login pages, event registration success pages and paginated lists only dilute the key points. List only the pages customers ask about: services, pricing, frequently asked questions and contact.
How to Check After Adding It: Four Steps to Test llms.txt
Each step takes one minute.
- Open it directly in a browser at
https://your-domain/llms.txt. You should see plain text content, with no garbled Chinese characters, not a 404, and not a redirect to the home page. - Check the returned headers. Run
curl -sI https://your-domain/llms.txton the command line and check whether the first line shows200and whetherContent-Typeistext/plainortext/markdown. - Click every link. Every link in the file must open.
- Run it through a tool. Chrome's Lighthouse has an experimental Agentic browsing category (requires Chrome 150 or above) that reads llms.txt, so one run tells you whether the file can be read normally. Then scan once with the free AEO Auditor to check robots.txt, structured data and AI crawler access at the same time.
Adding It Does Not Mean You Will Be Cited Tomorrow
llms.txt is a reading guide, not an admission ticket. Whether AI cites your website depends on whether crawlers can get in, whether the content has facts that can stand alone as sentences, and whether structured data is complete. llms.txt is only the last link in this chain. The numbers will not change right after you add it, and that is normal.
To find out which link in the chain is failing on your site, run a check with a free tool:
Check your website's AI visibility for free now
AEO here means Answer Engine Optimisation. It has nothing to do with the Authorized Economic Operator Programme of the Customs and Excise Department.
Frequently Asked Questions
Should Hong Kong Websites Add llms.txt?
Websites with several service pages, a knowledge base, or both Chinese and English versions should add one. Small shop websites with only a few pages can wait. Google Search itself does not use the file, and adding it neither helps nor harms rankings.
Where Should llms.txt Be Placed?
Place it in the domain root, that is your-domain/llms.txt. The second version of the specification allows a subpath, but the Chrome Lighthouse audit recommends the root directory. Each subdomain is independent and needs its own file.
How Do I Verify llms.txt After Adding It?
Open it directly in a browser and confirm it is plain text with no garbled Chinese characters. Use curl to check that it returns 200 and the Content-Type. Click every link. Then run the Agentic browsing category in Chrome Lighthouse once.
For more questions, see the tool's FAQ page.
Sources
Accessed: 2026-10-17
- llms.txt specification page (Jeremy Howard, Answer.AI, published 2024-09-03): https://llmstxt.org/
- Google Search Central, "AI features and your website" (updated 2026-07-10): https://developers.google.com/search/docs/fundamentals/ai-optimization-guide
- Search Engine Journal report on John Mueller's reply on Reddit in 2025-04 (2025-04-17): https://www.searchenginejournal.com/google-says-llms-txt-comparable-to-keywords-meta-tag/544804/
- Chrome for Developers, Lighthouse "llms.txt" audit documentation (updated 2026-05-05): https://developer.chrome.com/docs/lighthouse/agentic-browsing/llms-txt
- OpenAI, "Overview of OpenAI Crawlers": https://developers.openai.com/api/docs/bots
- Anthropic Help Centre, "Does Anthropic crawl data from the web, and how can site owners block the crawler?": https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler
- Perplexity, "Perplexity Crawlers": https://docs.perplexity.ai/docs/resources/perplexity-crawlers
- HTTP status and Content-Type of each site's llms.txt: tested with curl by the author of this article on 2026-10-17 against developers.openai.com, docs.anthropic.com, developers.cloudflare.com, developer.chrome.com/docs/devtools and www.ud.hk