Big Tech's AI Licensing Report Card Is In: What Publishers Should Actually Do Now

The State of AI Licensing: A Report Card Nobody Aced

Digiday just published their scorecard on Big Tech's AI licensing practices, and the results tell a story every publisher already suspected. Eight publishers rated the major AI platforms on criteria including transparency, willingness to pay, traffic impact, and crawler behavior. The verdict from one exec summed it up nicely: "All of them could be doing more. No one gets a great grade."

Microsoft came out on top with an 8/10 aggregate score. The platform has positioned itself as the collaborative partner publishers have been waiting for, complete with a pay-per-use model through its AI content marketplace. OpenAI landed second with a 7/10, earning points for its 18 licensing deals but losing them for ghosting smaller publishers.

Google, meanwhile, scored a dismal 2/10. Shocking absolutely no one who has watched AI Overviews cannibalize search traffic this year.

The AI Crawler Problem Nobody Wants to Talk About

The scorecard reveals something publishers have been dealing with quietly for months. AI crawlers range from "well-behaved" to "absolute nightmare," and the platforms making the loudest ethical claims sometimes run the worst bots. Understanding the difference between AI training crawlers and traditional SEO crawlers is essential for making informed blocking decisions.

Anthropic earned a perfect zero across every category. Publishers described its crawler as "a nightmare" that routinely flouts robots.txt requests. One exec noted they're "totally unresponsive" to any licensing partnership requests.

Perplexity fared only marginally better with a 1/10 on crawler behavior. According to publishers in the report, "Every time you take the mask off some crawler that's doing something abusive, you find out it's, once again, Perplexity under the hood." The company has allegedly used headless browsers to scrape content while denying the accusations.

Google's crawler situation remains complicated. The platform still hasn't fully separated its AI and search crawlers, which means publishers face an impossible choice. Block the AI training and you might hurt your search rankings. Allow it and your content feeds the machine taking your traffic. For specific guidance on this challenge, see our guide on how to block Google AI Overview from using your content.

Most Publishers Are Still Left Out in the Cold

The scorecard highlights a fundamental problem with the current licensing landscape. Microsoft has 11 publisher partners. OpenAI has 18. Meta just signed seven deals a few weeks ago. These numbers represent a tiny fraction of the publishing industry.

The vast majority of publishers have no seat at the licensing table. Several execs told Digiday their outreach to Microsoft has gone unanswered despite the platform's collaborative reputation. OpenAI apparently ignores publisher calls regularly. The current legal landscape around AI scraping remains uncertain, leaving most publishers without clear recourse.

This creates a two-tier system. Major publishers negotiate licensing deals worth millions. Everyone else watches their content get scraped without compensation while traffic evaporates to AI-generated summaries.

What Publishers Can Actually Do About AI Crawlers

The licensing discussion dominates headlines, but most publishers need practical protection strategies today. Waiting for a licensing deal that may never materialize is a losing proposition. Our comprehensive guide on how to block AI from scraping your website provides step-by-step implementation details.

The foundation of any AI protection strategy starts with your robots.txt file. This requires knowing which AI crawlers to block and understanding that new ones appear regularly. For detailed syntax and user agent strings, see our complete guide to blocking AI bots with robots.txt.

The major AI crawlers publishers should address:

• GPTBot: OpenAI's crawler for training and ChatGPT
• Google-Extended: Google's AI training crawler (separate from search, supposedly)
• ClaudeBot: Anthropic's crawler (the "nightmare" mentioned above)
• CCBot: Common Crawl, used by multiple AI companies
• Bytespider: TikTok's crawler with AI training implications
• Amazonbot: Amazon's crawler for Alexa and AI products

For a complete directory with ready-to-use blocking code, see our complete list of AI crawlers and how to block each one.

Effective blocking requires a multi-layered approach:

• Robots.txt rules: Declare which crawlers cannot access your content
• Meta tags: Add noai and noimageai directives to prevent content use in AI training
• Server-level blocking: Use your CDN or firewall to block known AI crawler IP ranges (our Cloudflare configuration guide covers this in detail)
• Rate limiting: Throttle suspicious traffic patterns that might indicate undeclared scrapers

The challenge is that blocking effectiveness depends entirely on whether the crawler respects your wishes. Microsoft and OpenAI generally comply with robots.txt. Anthropic and Perplexity, according to publishers in the Digiday report, frequently do not.

The Alternative: Optimizing for AI Citations Instead

Blocking isn't the only strategy. Some publishers are taking the opposite approach, optimizing their content to earn AI citations and capture emerging referral traffic. This approach makes sense for publishers whose content benefits from AI visibility.

The key question is whether blocking AI training crawlers actually hurts your AI referral traffic. The answer depends on your specific situation. Our decision framework for publishers can help you evaluate the trade-offs based on your content type, traffic sources, and revenue model.

Maximizing Revenue From the Traffic That Remains

The AI licensing scorecard conversation ultimately circles back to revenue. Publishers are evaluating platforms based on willingness to pay because content monetization is the point. Understanding how AI crawling affects your ad revenue helps quantify what's at stake.

The uncomfortable reality is that most publishers will never see licensing revenue. The platforms making deals are selective. Meanwhile, the traffic impact from AI products continues eroding pageview-based advertising revenue. For a deeper look at these economics, see our analysis of the real cost of blocking AI.

This makes protecting and maximizing your existing traffic more critical than hoping for licensing salvation. Every visitor who reaches your site represents monetizable attention that AI summaries cannot capture. The strategic question isn't whether Microsoft or OpenAI is the better licensing partner. It's how to ensure your ad revenue doesn't collapse while the industry sorts out who pays for what.

Playwire helps publishers navigate this landscape with up-to-date education on AI crawlers and tools while helping them maximize ad revenue from their remaining traffic. Waiting for Big Tech to figure out fair compensation has never been a revenue strategy.

Check your site's AI crawler protection status with our free AI Crawler Protection Grader, and explore our complete AI Crawler Resource Center for publishers facing these challenges.