Strategy

A third of publishers are blocking the AI crawlers. Should you?

Blocking Google's AI crawler protects your content from being summarized — and erases you from the answers buyers now read first. How to think about the tradeoff.

· 6 min read · by the Crescendo team

It’s the most tempting button in SEO right now: block the AI crawler, stop Google from summarizing your work and handing the answer to someone who never visits. Reporting in 2026 suggests as many as a third of publishers are moving to block AI Overviews in some form. The frustration is legitimate. The button is also, for most businesses, a trap.

What blocking actually does

First, the mechanics, because they’re widely misunderstood. Google ties AI Overview inclusion to its normal Search index. Historically you couldn’t cleanly opt out of Overviews while staying in regular search — the levers (like nosnippet) are blunt and also suppress the ordinary snippets that earn your remaining clicks. Blocking the broader AI crawlers (the ones feeding ChatGPT, Perplexity, and the rest) is cleaner, but it does exactly what it says: it removes you from the answers those engines generate.

So the real choice isn’t “protect my content vs. get scraped.” It’s “be a candidate to get cited vs. be absent.” Blocking doesn’t make the AI answer go away. It makes the AI answer about your category get written entirely from your competitors.

Who should consider it anyway

Blocking is a defensible call in a narrow band of cases:

  • Your business model is paid content. Subscriptions, licensed data, premium research — if AI summaries directly cannibalize what people pay for, the math can favor blocking. This is why some large publishers are doing it; their product is the article.
  • You have real leverage. If you’re a must-cite authority, withholding access is a negotiating position. Most brands aren’t in that seat and are fooling themselves if they think they are.

Who shouldn’t — which is most of you

If you’re a business that sells a product or service and uses content to get discovered, blocking is self-harm. Your buyers are asking assistants “what’s the best option for X” and acting on the answer. Removing yourself as an eligible source doesn’t protect anything — there was no paywall to protect — it just guarantees you lose every one of those moments to a competitor who stayed in.

The content was always a means to being chosen. AI answers are now where a lot of the choosing happens. Blocking the crawler to “keep” content that exists to win customers is optimizing the means over the end.

The better move than blocking

If the fear is “they take my work and give nothing back,” the answer isn’t to disappear — it’s to make sure that when the engine uses your work, it names you, and that the naming sends the buyer somewhere that converts. That’s the whole game of GEO: engineer the pages so you’re the cited source, not the uncredited training data.

And before you touch any of this, measure it. Find out whether you’re currently cited or invisible on your priority questions — blocking when you’re already winning citations would be lighting money on fire. Run the citation check first, then decide. The button will still be there. You’ll just have a reason to leave it alone.

More from the blog