🔍 Read the full analysis: What Does The AI Training Opt-Out Actually Cover? on ThorstenMeyerAI.com
Get business pricing on office and shipping supplies
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
OpenAI, Google, Microsoft and Anthropic have said they will support a shared robots.txt signal asking them not to use a website’s content to train generative AI models. The signal is voluntary, is being adopted on different schedules and does not block crawling, remove content from existing models or opt a site out of AI answers using live web content.
OpenAI, Google, Microsoft and Anthropic said on October 30 that they will honor a shared robots.txt signal asking AI companies not to use a website’s content for generative AI model training. The common rule could let publishers state that preference once instead of maintaining separate training directives for each participating company, but it is a voluntary request—not a technical block—and the companies will adopt it on different timelines.
The proposed rule uses the user-agent name __usage_ai-training, followed by Disallow: /. A website adding the lines would communicate a preference against its content being used to train generative AI models. The signal appears in an individual Internet-Draft titled “Machine-Readable Training Preference Signal,” written by Anthropic standards lead Kevin Kelley and published on October 30.
The companies have made different commitments. Anthropic says it has started implementation, though it says support will take time to reach its services. OpenAI aims to implement the signal “in the coming months.” Google says it aims to support it in 2027, with its crawler documentation specifying “early 2027.” Microsoft will honor it from February 1, 2027, as it crawls websites.
Microsoft also announced a separate page-level preference for AI Grounding: the use of web content to inform answers in Bing, Copilot and Microsoft 365. Its announcement says both preferences take effect on February 1, 2027. Microsoft says its older NoArchive and NoCache signals will stop counting as training or grounding preferences after March 1, 2027. The companies’ existing crawler controls—including directives for GPTBot, Google-Extended and ClaudeBot—remain in place during the change.
Infographic 01 · Who backs it, and when
One line in robots.txt, four companies, four timelines.
A shared signal against AI training. Only Anthropic says it has started; the others name dates or months. The standard itself is still a draft.
- 30 Oct 2026Draft published, four companies announce supportdoneIndividual Internet-Draft by Kevin Kelley (Anthropic)
- 30 Oct 2026Anthropic: “has already started implementing support”startedIts standards lead on the IETF list: “It’ll take us some time to add support across our services.”
- Coming monthsOpenAI aims to implement itplanExisting GPTBot and OAI-SearchBot rules keep applying
- 16 + 19 Nov 2026Working group meets at IETF 127, San FranciscoopenOn 21 Oct a chair said no consensus has been declared yet
- 1 Feb 2027Microsoft honors the new preferences as it crawlsplanTraining in robots.txt or per page; grounding per page
- 1 Mar 2027Bing stops reading NoArchive / NoCache as AI preferencesdeadlinePublish the new settings before this date
- 2027Google plans support; its docs say “early 2027”planA “no” maps to Google-Extended unless you set that token yourself
- 3 May 2027Draft version 00 expires unless updatedInternet-Drafts are valid for at most six months
Binding? No, voluntary
- OpenAI calls it a “voluntary effort”.
- The vocabulary draft does not “ensure that preferences are followed”.
- The draft “has no formal standing in the IETF standards process”.
- Private agreements may override it, the draft says.
Who else?
- Meta supports a common signal, but did not say whether it will honor it.
- Apple, Amazon, xAI, Perplexity: no statement in the sources reviewed for this article.
- The draft’s acknowledgments list contributors “for identification only”, not supporters.
Sources: IETF draft, 30 Oct 2026 · Anthropic on the IETF list · AIPREF minutes, 21 Oct 2026 · Google, 30 Oct 2026 · Google crawler docs · OpenAI, 30 Oct 2026 · Microsoft Bing, Oct 2026 · Meta (Andy Stone on X). Company plans are company statements.
Powered by Thorsten Meyer AI
For publishers, a common rule could reduce the work of expressing a training preference to several AI companies. Under the existing approach, site operators may need distinct instructions for each crawler or service. The new signal offers a shared format, but its practical effect depends on companies implementing and honoring it. It does not prevent a crawler from accessing a site in the way a technical access restriction might.
The distinction between training and other uses is important. The proposed shared signal addresses training generative AI models; it does not, by itself, cover content retrieved from the live web to answer a user’s question. Microsoft is providing a separate grounding preference, but the announcements do not establish that other companies will adopt a matching control. Microsoft also says neither choice guarantees inclusion, traffic or payment.
For readers and creators, the rule is about how publicly accessible website material may be used, not a setting for personal conversations with an AI assistant. It also cannot undo prior use: the signal does not remove material from models already trained. Those limits mean publishers need to distinguish training, AI answer features and search access when deciding which controls to use.
Popular right now
De'Longhi 15-Bar Pump Espresso Machine with Milk Frother & Temperature Control
De'Longhi Rivelia Automatic Espresso Machine with Milk Frother
Vitamix 5200 Professional-Grade Blender, 64 oz. Container, Black
As an affiliate, we earn on qualifying purchases.
From Company Rules to One Draft
Before this announcement, website operators generally had to consult each company’s own controls. OpenAI uses GPTBot rules, Google has Google-Extended, and Anthropic has ClaudeBot. Microsoft has used page-level NoArchive and NoCache tags for certain preferences. A shared rule is intended to simplify that patchwork, while leaving the existing controls available.
The proposal is connected to the AI Preferences working group of the Internet Engineering Task Force, which develops internet standards. However, publication as an Internet-Draft does not make the text an adopted standard. The source article reports that the draft has no formal standing in the IETF standards process; at a working-group meeting on October 21, one of the chairs said, “We haven’t declared consensus on anything yet.”
Meta’s communications chief, Andy Stone, said the company supports publishers’ ability to express preferences and called a common signal an important step. That statement did not commit Meta to honoring this particular rule. The source material also reports no implementation commitment from Apple, Amazon, xAI or Perplexity. The draft’s acknowledgments include people from technology companies, publishers and other organizations, but their inclusion does not establish that those organizations have adopted the signal.
““has already started implementing support for this draft””
— Anthropic
Adoption and Enforcement Remain Open
The signal is a voluntary preference, and the announcements do not specify a mechanism that would force a company to follow it. The companies’ stated schedules also differ, so a site owner cannot assume the shared rule will receive the same treatment across services immediately. Anthropic’s stated implementation is underway, OpenAI’s timing is described as the coming months, and Google’s target is 2027.
Meta has voiced support for publishers expressing preferences but has not said whether it will honor this specific signal. The source material identifies no public commitment from several other major AI services. It is also not yet clear how widely the draft will be adopted, how the IETF process will proceed or whether additional companies will add support.
The scope is limited as well: the shared token addresses training, not every use of website material. It does not block crawling, control live-web grounding for services other than Microsoft’s separate preference, guarantee payment, or reverse training that has already occurred. Publishers should check each company’s documentation for the current effect of its existing controls.
Watch Company Rollouts Through 2027
The next concrete milestone is implementation by each company. Anthropic says it has begun; OpenAI has set a general target of the coming months; Google’s published target is early 2027; and Microsoft’s announced start date is February 1, 2027. Microsoft says its older NoArchive and NoCache preferences stop counting for training or grounding after March 1, 2027, making that date relevant for sites that rely on those controls.
Publishers considering the new rule can add the shared training preference while keeping existing company-specific crawler instructions until each provider’s implementation is in effect. They should separately decide whether they want to restrict training, AI grounding or search access, since those are not interchangeable choices. The draft’s status in the IETF process and any further company commitments remain developments to monitor.
Key Questions
What does the shared AI training opt-out do?
It adds a robots.txt preference asking participating companies not to use a website’s content to train generative AI models. The proposed user-agent is __usage_ai-training, followed by Disallow: /.
Does the signal block AI companies from crawling a website?
No. The source describes it as a request or preference, not a technical lock. It does not itself prevent a crawler from accessing the site.
Does opting out stop AI answers from using a site’s content?
Not under the shared training signal. It covers model training, not answers informed by live web content. Microsoft announced a separate AI Grounding preference for Bing, Copilot and Microsoft 365.
When will the companies honor the signal?
Anthropic says implementation has started; OpenAI aims for the coming months; Google targets 2027, with documentation saying early 2027; and Microsoft starts on February 1, 2027. Meta has expressed support for publisher preferences but has not committed to honoring this signal.
Is the shared signal an official internet standard?
No. It is an individual Internet-Draft associated with the IETF’s AI Preferences working group. The source says it has no formal standing in the standards process, and the group had not declared consensus.
Source: ThorstenMeyerAI.com
