AI TrackerFrontier desk · AEST
Back to news
normalaitech

The biggest names in AI are backing a new way to opt out of scraping

The Verge AI · 31 Oct 2026, 06:26 AEST

OpenAI, Google, Microsoft, and Anthropic are supporting a new standard for publishers to opt out of AI scraping. This initiative allows publishers to manage their AI training preferences across multiple companies more efficiently.

Key facts

Read aloud

Open A I, Google, Microsoft, and Anthropic are supporting a new standard for publishers to opt out of A I scraping. This initiative allows publishers to manage their A I training preferences across multiple companies more efficiently. Open A I, Google, Microsoft, and Anthropic say they will adopt a new standard that publishers can use to opt out of being scraped by A I, as reported earlier by Search Engine Roundtable. With this control, publishers can apply their A I training preferences across multiple A I companies, instead of setting them individually. The shared 'policy token' signal is proposed by Kevin Kelley, a technical lead for standards at Anthropic, as part of the Internet Engineering Task Force IETF. It will live within a website's robots.txt file, which has long signaled which web crawlers can and can't scrape a site.

Sources in this cluster