Four leading tech giants including OpenAI and Google have unified AI training opt-out rules, removing the need for websites to set up separate rules for each platform.

Beating AI News Brief: OpenAI, Google, Microsoft, and Anthropic have reached an agreement: going forward, websites only need to add two rules to their `robots.txt` file — `User-Agent: __usage_ai-training` and `Disallow: /` — to notify all four companies simultaneously not to use their content for AI training. Previously, websites had to set rules separately for crawlers like GPTBot and ClaudeBot. This rule set is exclusively for AI training, does not block crawlers from accessing websites entirely, and does not necessarily prevent AI from citing website content in its responses. Proposed by an Anthropic engineer, the framework is currently a technical draft from the Internet Engineering Task Force (IETF). The companies will not implement the rules uniformly: Microsoft allows websites to block AI training specifically while keeping their content eligible for citation in AI search results; Google plans to tie this rule to its existing Google-Extended setting, which may also restrict Gemini from citing website content. Anthropic has already begun adopting the rule, Microsoft plans to roll out its implementation in February 2027, Google aims to support it in 2027, and OpenAI will launch its version in the coming months. However, these rules cannot reverse data already used for prior training, nor can they force AI companies to comply.