buffy.blog
Field note

Google AI controls compared: robots.txt, Google-Extended, nosnippet, the AI opt-out, and noindex

Five different controls decide how Google's AI features and crawlers treat your site, and they are constantly confused for one another. This neutral, criteria-based reference table lays out exactly what each one does — and does not do — to normal Search ranking, AI Overviews, AI Mode, Gemini training, and featured snippets, so you pick the right lever instead of the blunt one.

Buffy Editorial2026-08-20 · 4 min read

Five different Google controls decide how your content is crawled, ranked, shown in AI answers, and used for training — and they do genuinely different jobs that are easy to mix up. Reaching for noindex when you meant the AI opt-out, or assuming Google-Extended keeps you out of AI Overviews, is a common and costly mistake. This is a neutral, criteria-based reference: what each lever does, what it leaves alone, and when it is the right one.

Last reviewed: 20 August 2026. Behaviour and availability — especially of the newest control — are moving; the descriptions below reflect Google's stated guidance and reputable reporting as of mid-2026. Verify against current Search Console and Search Central documentation before acting.

The five controls at a glance

Each row is one lever; each column is a thing you might want to protect or remove. Read across to see exactly what a control touches.

Control Normal Search ranking/indexing AI Overviews & AI Mode Featured / text snippets Gemini model training Scope
Search generative AI control (Search Console setting) Unaffected Removes you Unaffected Unaffected Google AI surfaces only
nosnippet / data-nosnippet / max-snippet:0 Page still ranks Removes you Also removed Unaffected The specific page/element
Google-Extended (robots.txt token) Unaffected Not removed on its own Unaffected Limits training + Gemini-app grounding Gemini training/grounding
Block Googlebot (robots.txt) Not crawled → drops out Removed (not crawled) Removed Removed Everything Google
noindex (meta/header) Removed from index Removed Removed Removed Everything Google

Source: Google's stated guidance and reputable SEO reporting, mid-2026. The pattern to notice: only the Search generative AI control removes you from the AI answer while leaving normal ranking and snippets intact. Every other "remove me" lever takes something else down with it.

Only one of Google's five controls removes you from the AI answer without also costing you ranking or snippets — reach for the wrong lever and you take down far more than you meant to.

Which control should you use for each goal?

Match the lever to the outcome, not to the fear. The precise goal decides the tool.

  • "Stay in Search but out of AI Overviews/AI Mode." Use the Search generative AI control. It is the only surgical option, and it is reversible with a toggle.
  • "Stop my content training Gemini, but keep appearing normally." Use Google-Extended in robots.txt. Note it governs training and Gemini-app grounding — not whether you show in AI Overviews.
  • "Keep this specific preview text out of results." Use nosnippet or data-nosnippet on the element — accepting that it also removes featured-snippet eligibility for that page.
  • "Remove this page from Google entirely." Use noindex. Expect to lose ranking, snippets, and AI features together.
  • "Stop Google crawling a section at all." Block the path for Googlebot in robots.txt — the bluntest option, with the widest collateral effect.

Why do people pick the wrong lever?

Because the controls sit at different layers — crawl, index, display, and training — and the labels do not make that obvious. Three recurring mistakes:

  1. Assuming Google-Extended blocks AI Overviews. It does not on its own; AI Overviews are built from Google's Search index, so a page Googlebot can still crawl and index can still surface in the AI answer even with Google-Extended set. The training-versus-retrieval distinction is the crux — Google-Extended is a training/grounding control, not a retrieval-display control.
  2. Using nosnippet to leave AI Overviews and then losing featured snippets. This was the only pre-2026 route and it quietly cost classic-Search visibility; the Search generative AI control now removes that trade-off.
  3. Reaching for robots.txt or noindex to make one AI change. Both are sledgehammers that also demolish your normal Search presence. For an AI-only decision they are almost never the right tool.

Do these controls affect other AI engines?

No — and this is the biggest scope error of all. Every lever in the table is a Google control. None of them changes how ChatGPT, Perplexity, Claude, or other engines crawl, retrieve, or cite your pages. Those engines are governed by their own AI crawler directives in your robots.txt and their own retrieval systems. Deciding whether to be open or closed to those engines is a separate, cross-engine strategy question — see whether to let AI crawlers index your site and, at the more adversarial end, blocking or licensing AI crawlers. The standing default for a visibility-seeking brand is to keep AI crawlers reaching your site and win the citation, not to wall it off.

Picking the right control is only useful if you can see the result. Buffy Intel tracks your brand's presence across Google's AI Overviews and AI Mode alongside ChatGPT, Perplexity, and Claude — daily — so you know whether a lever did what you intended. Check your AI visibility with Buffy Intel, or reach us at [email protected].

Frequently asked

What is the difference between Google-Extended and blocking Googlebot?

They act at different points. Blocking Googlebot in robots.txt stops Google from crawling the page at all, which cascades into losing normal ranking, snippets, and AI features together — a blunt instrument. Google-Extended is a separate robots.txt token that only limits use of your content for training Gemini models and grounding in the Gemini apps; it does not remove you from normal Search and, on its own, does not remove you from AI Overviews. One controls whether Google sees the page; the other controls one downstream use of what it saw.

Does noindex remove my page from AI Overviews?

Yes, but as a side effect of removing it from everything. noindex tells Google not to index the page, so it drops out of the entire index — normal results, snippets, and AI features all at once. It is the most complete removal and the least surgical: you would only use it when you do not want the page in Google Search at all, not when you simply want to stay out of the AI answer while keeping your blue-link ranking.

Which control keeps my ranking but removes me from AI answers?

The Search generative AI control in Google Search Console, introduced in mid-2026. It removes a site from AI Overviews, AI Mode, and Discover's AI features while leaving normal Search ranking and indexing untouched, and Google has said it is not used as a ranking signal. Before it existed, the closest option was the nosnippet directive, but that also stripped featured snippets and preview text — a coupled cost the new control avoids.