Field note

How to diagnose and fix pages Google crawls but won't index

A five-step diagnostic for pages stuck in Google's 'Crawled – currently not indexed' status: confirm which status you have, rule out the technical causes that hide your content, recognise the pages that belong there, then judge honestly whether the real problem is commodity content, and fix it. Getting indexed is the entry ticket to AI Overviews and AI Mode.

Buffy Editorial2026-07-17 · 5 min read

Pages stuck in Google's "Crawled – currently not indexed" status are pages Google downloaded and chose not to keep. They cannot appear in Google Search, and because AI Overviews and AI Mode draw only from the index, they cannot be cited there either. This is a five-step diagnostic to find the cause and fix it, working from the quick technical checks to the harder quality question.

It is the practical companion to crawled but not indexed: why Google's indexing bar is now your AI-visibility bar. That piece is the why; this is the procedure.

Rule out the technical causes first because they are fast to check. But be honest: for most stuck pages, the real answer is that the content is a commodity, and no technical fix will change that.

Step 1: Confirm which status you actually have

Open Search Console → Indexing → Pages and check the exact label. "Crawled – currently not indexed" and "Discovered – currently not indexed" are different problems:

  • Discovered – currently not indexed: Google knows the URL but hasn't crawled it yet. This points to crawl-budget, internal-linking, or scheduling issues, not a quality verdict.
  • Crawled – currently not indexed: Google fetched the page and declined to index it. This is either a technical fault hiding your content or a quality judgment.

The rest of this guide is about the crawled case. Getting the label right stops you chasing the wrong fix.

Step 2: Rule out the technical causes that hide your content

These are rarer than quality problems but faster to check, so do them first. Use URL inspection in Search Console: click the magnifying glass next to a stuck URL, then Test Live URL, and look at the rendered HTML and screenshot.

  • Does Google see real content? If the live test shows only a heading and a few boilerplate words, something is stopping Google from rendering the page. If it shows the full content, the cause is almost certainly quality, not technical (skip to Step 4).
  • Is robots.txt blocking your assets? A rule meant to block tracking parameters, such as Disallow: /*?*, can also block parameterised CSS and JavaScript files a theme depends on. Google then sees an unstyled, contentless shell. Check that your robots.txt isn't blocking resources the page needs to render.
  • Is the content locked in JavaScript? Confirm your facts are in server-rendered HTML, not injected client-side. See do AI assistants render JavaScript? — the same rendering gaps that hurt AI engines also cost you in Google's index.
  • Are you accidentally blocked? Rule out a noindex tag, a canonical pointing elsewhere, or a CDN blocking crawlers.

If Google can see a full page and nothing is blocking it, stop looking for a technical cause.

Step 3: Recognise the pages that belong in this bucket

Not every crawled-not-indexed URL is a problem. Some pages should be there, and trying to force them in wastes effort:

  • Non-canonical duplicates (Google is correctly consolidating to the canonical).
  • Pagination pages (/page/2/ and beyond).
  • Feed URLs (/feed/) and archive variants.
  • URLs with tracking or filter parameters.

Filter these out before you count your problem. If the stuck pages are all parameter variants and feeds, you may have no issue at all. Focus only on canonical, content-bearing pages you genuinely want indexed.

Step 4: Judge honestly whether the content is a commodity

When the page is technically fine and canonical, the cause is almost always quality. The pattern reported from Google's mid-2026 guidance is commodity content: pages that restate what many others already cover without adding anything more useful. Google described wanting to index content that offers two things it can't get elsewhere: first-hand experience and knowledge no one else has.

Audit each stuck page against that bar, plainly:

Ask If the answer is "no"
Does it contain first-hand experience, testing, or original data? It reads as generic and easily replaced
Does it say something the top existing pages don't? It is a rehash competing with better-established pages
Would a reader who saw the other results still find this useful? It adds no reason to be indexed

This is the same standard AI engines apply when they decide what to cite. A page too generic for Google's index is usually too generic for an AI answer.

Step 5: Fix the cause, then validate

Match the fix to the cause you found:

  1. Technical fault: remove the block (fix robots.txt, the render, the canonical), then use Validate Fix so Google re-crawls. Recovery is gradual, over weeks.
  2. Commodity content: consolidate several thin pages into one strong, comprehensive page; add first-hand experience, original data, and specifics; or retire pages that duplicate stronger ones. A substantive rewrite beats a date bump past the freshness cliff.
  3. Site-wide pattern: if a large share of your pages are stuck, treat it as a content-quality problem across the site, not a page-by-page cleanup, and prioritise your highest-value URLs first.

A fair warning from the same guidance: sites with many pages stuck in this bucket often find recovery difficult, because the cause is systemic. There is no shortcut around producing content worth indexing.

The whole loop in one line: confirm the status, rule out the fast technical causes, ignore the pages that belong there, and spend your effort making the rest worth indexing, because the pages Google will index are increasingly the pages AI engines will cite. Watching which of your pages are indexed, reachable, and actually cited across every AI surface, over time, is exactly what Buffy Intel is built to do.

Frequently asked

How do I fix 'Crawled – currently not indexed' pages?

Work in order. First confirm the status is 'Crawled' not 'Discovered'. Then rule out technical causes: use URL inspection and Test Live URL to check Google actually sees your content, and check robots.txt isn't blocking the CSS or JavaScript your page needs to render. If Google can see the content, the cause is almost always quality: the page restates what many others already cover. Fix that by consolidating thin pages, adding first-hand experience and data, or accepting that some pages don't need to be indexed.

How long does it take to recover a crawled-not-indexed page?

Recovery is slow and not guaranteed. Once a technical block is removed, pages tend to return to the index gradually over weeks, not days. When the cause is quality, improving one page may help, but a site with a large number of pages stuck in the bucket can be hard to fully recover, because the underlying issue is often site-wide commodity content rather than one weak page. Set expectations accordingly and prioritise your highest-value URLs.

Should I use the 'Validate Fix' button in Search Console?

Only after you have actually made a change that addresses the cause. 'Validate Fix' asks Google to re-check the affected pages; clicking it without fixing anything just restarts a validation that will fail. Rule out the technical issue or improve the content first, confirm with a live URL test that Google can now see a substantive page, then validate so Google re-crawls and re-evaluates.