Crawlers & Technical SetupPart 6 of 13

Is llms.txt worth it? An honest take

llms.txt is a tidy idea. A curated map of your site for AI. But as of 2026 the major engines don't broadly honour it. Here's the honest verdict: low-cost hygiene, not a visibility lever.

Buffy Editorial2026-06-09 · 2 min read

llms.txt is a tidy idea: a Markdown file at your site root that hands AI a curated map of your most important pages. It's easy to be enthusiastic about. A robots.txt for the AI era. So here's the honest take, because over-promising on it is a disservice.

The optimistic case

The pitch is reasonable. Instead of making an engine infer your site's structure from messy HTML, you tell it plainly: here are the key pages, here's what each is. For an AI trying to understand a site quickly, a clean, curated guide is genuinely helpful in principle.

The reality (as of 2026)

It's a proposed convention, not an honoured standard. The major AI engines don't broadly read or act on llms.txt today, and there's no solid evidence it moves citations on its own. Adoption among sites is growing, but adoption by the engines that matter is the part that counts, and that hasn't arrived. Treat any claim that "adding llms.txt boosted our AI visibility" with skepticism; correlation in a fast-moving space is easy to misread.

llms.txt is a bet on where the ecosystem is going, not a lever that works today. Add it like you'd add a sensible standard early, not like a growth tactic.

So should you add it?

Yes, but calibrate your expectations. It's near-zero cost, has no downside, keeps you positioned if engines start honouring it, and signals you take AI readability seriously. Add it, keep it accurate, and move on.

What you should not do is treat it as a substitute for the things that actually drive AI visibility today:

  • Crawl access: AI bots can reach you (robots + CDN both open).
  • Parseability: clean, semantic, answer-first content in real HTML, not JS-only.
  • Structured data: schema that labels what your content is.
  • Authority & freshness: a strong entity, corroborated and kept current.

The honest bottom line

Add llms.txt. Then forget about it and go do the work that moves the needle. The brands winning in AI search aren't winning because of a text file. They're winning because they're reachable, legible, authoritative, and measured. (We added one to this site for exactly this reason: cheap, future-proofing, not a crutch.)

Frequently asked

Does llms.txt actually improve my AI visibility?

There's no good evidence it moves citations on its own yet. As of 2026 it's a proposed convention that the major AI engines don't broadly act on. It's worth adding as cheap, future-proofing hygiene, but the real visibility levers are crawl access, semantic structure, schema, freshness, and authority, not an llms.txt file.

Should I add an llms.txt file anyway?

Yes. It's near-zero cost, carries no downside, and positions you if engines start honouring it. Just keep it accurate and don't expect it to do heavy lifting. Spend your real effort on being reachable, parseable, and authoritative.