AI TrackerFrontier desk · AEST
Back to news
normaltech

Google, OpenAI, Microsoft, and Anthropic back a shared robots.txt

Hacker News · 3 Nov 2026, 04:11 AEST

Google, OpenAI, Microsoft, and Anthropic are supporting a shared robots.txt initiative. This collaboration aims to create a standardized way for web crawlers to interact with websites.

Key facts

Read aloud

Google, Open A I, Microsoft, and Anthropic are supporting a shared robots.txt initiative. This collaboration aims to create a standardized way for web crawlers to interact with websites. Google, Open A I, Microsoft, and Anthropic back a shared robots.txt Google, Open A I, Microsoft, and Anthropic back a shared robots.txt. Google, Open A I, Microsoft, and Anthropic back a shared robots.txt

Sources in this cluster