AI crawlers do not follow links that only exist after JavaScript executes. In a mid-2026 field experiment, SEO engineer Vinicius Stanula split a roughly 1,000-page site so half its navigation links sat in the raw HTML and half were injected by JavaScript, then logged what 27 bots did over 41 days. Only Google's two crawlers followed the JavaScript links. Every AI crawler tested. GPTBot, ClaudeBot, PerplexityBot, and the rest. Reached zero pages that were reachable only through a script-generated link. The practical lesson is blunt: a link a bot can't see in the HTML source is, for that bot, a link that doesn't exist.
This is part of the crawlers and technical series and the field-data companion to our piece on whether AI assistants render your JavaScript. That piece asked whether an engine reads script-injected content; this one asks something upstream. Whether a crawler can even discover a page when the only route to it is built by JavaScript.
What did the JavaScript-link experiment actually test?
It isolated link discovery from everything else. Stanula took a site with 21 top-level categories and about 1,000 pages beneath them, then split the categories in two: 11 kept normal HTML links in the page source, and 10 had their links injected by JavaScript so they were invisible in the raw markup. Same design, same content, same templates. Each subcategory inherited its parent's condition. The only variable was whether the path to a page existed before or after scripts executed.
The experiment lasted 41 days: 28 days with the split live, then 13 more after every link was converted back to plain HTML, to watch recovery. Edge middleware logged 30,180 bot requests across 27 distinct crawlers that reached the surface pages. Because the surface (linked-from-homepage) pages were reachable by everyone, the test cleanly measured one thing: which bots could get past a JavaScript link to the pages behind it.
Which crawlers followed the JavaScript links?
Almost none. Of the 27 bots that reached the site, only Googlebot and GoogleOther: 2 of 27. Followed the JavaScript-injected links to the pages behind them. Every other crawler, including all the AI crawlers, found nothing there.
| Crawler | Followed JavaScript links? | Pages found behind JS |
|---|---|---|
| Googlebot / GoogleOther | Yes | Followed (with depth loss, below) |
| GPTBot (OpenAI) | No | Zero |
| ClaudeBot (Anthropic) | No | Zero |
| PerplexityBot | No | Zero |
| Bingbot (Microsoft) | No | Zero |
| Meta, Amazonbot, DuckDuckBot, Yandex, others | No | Zero |
GPTBot was an especially clear case: it made 3,809 requests to the site and crawled zero pages that lived behind a JavaScript link. All figures are attributed to Stanula's experiment; it is a single test on one site, so read the specific counts as directional, not a benchmark.
Even Google lost pages as the links went deeper
Google could follow the JavaScript links, but coverage decayed sharply with depth. On the HTML side of the site, Google crawled 100% of pages at every level. All 759 of them. On the JavaScript side, it reached 67% of pages one level down, 56% two levels down, and 42% at the deepest level.
Speed and coverage turned out to be different problems. Google found its first JavaScript page in about 7.5 hours, close to the 6.1 hours it took on the HTML side, so the JavaScript links weren't slow to process, they were incompletely followed. Fast discovery of some pages masked the fact that many never got crawled at all. For engines that don't execute scripts, there is no partial credit: the coverage is simply zero.
A link a crawler can't see in the HTML source is a link that doesn't exist. For 26 of the 27 bots in this experiment. Every AI crawler among them. A JavaScript-injected link led nowhere.
What happened when the links were switched back to HTML?
Recovery was fast for the crawlers that had been shut out. In the 13 days after every link was converted to plain HTML, GPTBot picked up 250 new pages: the first 250 within 48 hours, and Bingbot found 89. Google, the one crawler that could already render the pages, added zero: it had nothing new to discover.
That asymmetry is the whole story in miniature. The AI crawlers had been blind to those pages the entire time, not slow. The moment a plain-HTML path appeared, they rushed in. The pages hadn't changed; only their discoverability had.
Does this hold up beyond one experiment?
The pattern is well corroborated, even if one field test isn't proof on its own. Independent analyses through mid-2026 reach the same conclusion: AI crawlers fetch raw HTML and do not execute JavaScript. One analysis of more than 500 million GPTBot fetches reported no evidence of script execution. GPTBot downloads JavaScript files a fraction of the time but doesn't execute them (figures reported by Lantern, mid-2026, single-vendor and directional). Our own coverage of a 12-engine rendering test found the major Western assistants read only the raw HTML. The consistent exception is Google, whose crawlers share the rendering infrastructure behind Googlebot, which is exactly what this experiment shows.
The takeaway is durable even as individual numbers move:
- Put real links in server-rendered HTML. Navigation, pagination, related-content, and "load more" links should be standard
<a href>anchors present before any script executes. - Give every important page a plain-HTML path from a page that is itself crawlable. Don't strand pages behind client-side routing.
- Verify from the source, not the inspector. View Source or
curlthe page; if a link only appears after the browser executes code, the AI crawlers won't see it. - Don't confuse rank with reach. A site can rank well on Google, which renders, while remaining invisible to the AI crawlers that decide which brand gets cited.
Reachability isn't only about whether a bot is blocked at your CDN or allowed in your robots file. It's about whether your links survive the trip from fetch to parse. Confirming that every citable page has a plain-HTML path the AI crawlers can actually follow is exactly the kind of check Buffy Intel keeps watch on. Questions: [email protected].