On Google's Search Off The Record podcast (Episode 111, discussed June 2026), John Mueller explained the fundamental flaw of llms.txt: because the site owner writes it about their own site, an AI system is designed not to trust it as a way of telling one website from another. A signal you fully control is a weak signal, which is why, in Google's view, llms.txt can't do the AI-discovery job many publishers hope it will. The reporting comes from Search Engine Journal's write-up of the episode; Mueller's position is Google's, and other engines may treat the file differently.
This is part of the crawlers and technical series and a direct companion to our earlier honest take on whether llms.txt is worth it. That piece gave the verdict. Cheap hygiene, not a lever. This one explains why, in Google's own framing.
What did Google say about llms.txt?
That you can't be trusted to grade your own homework. Mueller's argument is about the trust model of retrieval, not the file format. In his words on the podcast:
"Because it's basically you're telling these systems, like, I have the best website ever … So in an LLM system, it … basically, by design, can't trust what is here as a way of differentiating between different websites."
The logic is simple and durable. llms.txt is content the site owner authors about their own site. A curated list of "my most important pages." Every site owner has the same incentive to claim importance, so the file carries no information that separates a genuinely authoritative page from a self-promotional one. A retrieval system built to surface trustworthy evidence has to discount a signal that every publisher can set in their own favour.
Why is a file you control a weak signal?
Because search has been here before, with the keywords meta tag. Mueller drew the parallel explicitly, and it's the clearest way to understand the flaw. The <meta name="keywords"> tag let site owners declare what their page was about. Search engines stopped trusting it more than a decade ago for exactly one reason: it was owner-controlled and therefore trivial to manipulate. llms.txt sits in the same category.
| Property | Keywords meta tag | llms.txt |
|---|---|---|
| Who writes it | The site owner | The site owner |
| Verified against the page? | No | No |
| Standards body / enforcement | None | None |
| Incentive to inflate | High | High |
| How engines treat it | Ignored for 10+ years | Not broadly consumed (as of 2026) |
The pattern is the lesson: the more completely you control a signal, the less a retrieval system can rely on it. This is the same principle behind why coordinated, self-serving AI content gets discounted and why AI search resists manipulation. Trust in these systems is earned by corroboration from sources you don't control, not by assertions about yourself.
What was llms.txt actually designed for?
A narrower job than discovery. Mueller said the original idea was not to make it easier for engines or LLMs to discover all of your content. It was closer to the reverse: if an AI already knows about your site and is operating on it, llms.txt might help it navigate. That is a real but modest use, and a long way from "publish llms.txt so AI finds you."
Even for the navigation job, Mueller suggested that emerging agent standards. Such as WebMCP, one of the two paths to agentic commerce. Are better suited, because they're designed for an agent already interacting with your site rather than for discovery. The mismatch between what llms.txt was meant to do and what publishers use it for is the core of the confusion.
So should you delete your llms.txt?
No. Nothing here says the file is harmful. As the honest take concluded, it's near-zero cost, carries no downside, and positions you if engines ever start honouring it. Keep an accurate file if you have one; just don't expect it to move citations, and don't let it crowd out the work that does. The error is the expectation, not the file.
What actually earns AI trust?
The signals you can't fake on your own. Mueller's closing point is that genuine discovery and ranking still depend on your HTML pages. What's actually on the open, crawlable web, not on a proprietary file. That points straight back to the durable levers:
- Crawlable, semantic HTML. Discovery happens through the pages themselves. Make them reachable and parseable, not locked behind JS or a stripped separate "AI version".
- Structured data. Schema labels what your content is in a verifiable, standardised way. The opposite of a free-text self-claim.
- A corroborated entity. Being described consistently by sources across the web is the trust signal llms.txt can't be, because you don't author it.
- Freshness and depth. Current, specific, genuinely useful pages are what retrieval systems lift, and what keeps you on the right side of the citation freshness cliff.
The takeaway isn't anti-llms.txt; it's a reminder of how AI trust works. You earn it by being legible and corroborated, not by telling the system you're the best. Tracking whether engines actually cite you, and which of your pages they pull from. Is what Buffy Intel measures. Questions: [email protected].