🔍 Read the full analysis: A Step-by-Step Guide To Setting The AI Training Opt-Out on ThorstenMeyerAI.com
Get smart everyday buys delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
OpenAI, Google, Microsoft and Anthropic have said they will honor a shared robots.txt signal asking that website content not be used to train generative AI models, with different rollout schedules. The rule is voluntary, does not block crawling or AI answers based on live web content, and has not become an official internet standard.
OpenAI, Google, Microsoft and Anthropic have said they will honor a shared robots.txt signal asking that a website’s content not be used to train generative AI models. The proposal gives publishers one line to communicate that preference to participating companies, but it is voluntary, is not yet an official internet standard and will take effect on different schedules.
The signal appears in an Internet-Draft titled Machine-Readable Training Preference Signal, published on October 30 by Anthropic standards lead Kevin Kelley. To express the preference, a site operator adds this rule group to the site’s robots.txt file:
User-Agent: __usage_ai-training
Disallow: /
The rule is a request about training use, not a technical barrier that prevents a company from accessing a site. Existing company-specific controls, including rules for GPTBot, Google-Extended and ClaudeBot, remain relevant while providers implement the shared signal.
The companies have offered different timelines. Anthropic says it has started implementation, while adding that support will take time to reach its services. OpenAI aims to implement the signal in the coming months. Google says it aims to support it in 2027, and its crawler documentation specifies early 2027. Microsoft says it will honor the shared training preference from February 1, 2027, as it crawls sites.
Microsoft also announced a separate page-level preference for content used to ground AI answers in Bing, Copilot and Microsoft 365. That setting is distinct from the shared training signal. Microsoft says the new preferences will replace NoArchive and NoCache for training or grounding purposes after March 1, 2027. It says those controls do not change a page’s eligibility for Bing search results.
Infographic 01 · Who backs it, and when
One line in robots.txt, four companies, four timelines.
A shared signal against AI training. Only Anthropic says it has started; the others name dates or months. The standard itself is still a draft.
- 30 Oct 2026Draft published, four companies announce supportdoneIndividual Internet-Draft by Kevin Kelley (Anthropic)
- 30 Oct 2026Anthropic: “has already started implementing support”startedIts standards lead on the IETF list: “It’ll take us some time to add support across our services.”
- Coming monthsOpenAI aims to implement itplanExisting GPTBot and OAI-SearchBot rules keep applying
- 16 + 19 Nov 2026Working group meets at IETF 127, San FranciscoopenOn 21 Oct a chair said no consensus has been declared yet
- 1 Feb 2027Microsoft honors the new preferences as it crawlsplanTraining in robots.txt or per page; grounding per page
- 1 Mar 2027Bing stops reading NoArchive / NoCache as AI preferencesdeadlinePublish the new settings before this date
- 2027Google plans support; its docs say “early 2027”planA “no” maps to Google-Extended unless you set that token yourself
- 3 May 2027Draft version 00 expires unless updatedInternet-Drafts are valid for at most six months
Binding? No, voluntary
- OpenAI calls it a “voluntary effort”.
- The vocabulary draft does not “ensure that preferences are followed”.
- The draft “has no formal standing in the IETF standards process”.
- Private agreements may override it, the draft says.
Who else?
- Meta supports a common signal, but did not say whether it will honor it.
- Apple, Amazon, xAI, Perplexity: no statement in the sources reviewed for this article.
- The draft’s acknowledgments list contributors “for identification only”, not supporters.
Sources: IETF draft, 30 Oct 2026 · Anthropic on the IETF list · AIPREF minutes, 21 Oct 2026 · Google, 30 Oct 2026 · Google crawler docs · OpenAI, 30 Oct 2026 · Microsoft Bing, Oct 2026 · Meta (Andy Stone on X). Company plans are company statements.
Powered by Thorsten Meyer AI
What Publishers Can Control
The shared rule could make it simpler for publishers to state a preference without maintaining a separate training instruction for every participating company. That matters to site owners who want to communicate how their material may be used, especially while AI providers are adding web content to training and answer-generation systems.
But the signal should not be mistaken for a lock, a takedown request or a payment mechanism. It asks participating companies not to use covered content for training; it does not technically stop access, reverse use of material in models already trained, or guarantee that a site will receive traffic or compensation. Microsoft has said that its choices provide no guarantee of inclusion, traffic or payment.
Publishers also need to distinguish among model training, AI-generated answers and search. The shared signal addresses training. Microsoft’s separate grounding preference concerns content used to inform certain AI answers. The announcements do not establish that the shared rule governs every form of AI use, and the stated search policies remain separate.
Products Worth Considering
Dell UltraSharp 27 Thunderbolt Hub Monitor (U2726DER)
Dell 34-Inch S3425DW Curved WQHD USB-C Monitor
FLEXISPOT Standing Desk with Monitor Riser & Cable Management
As an affiliate, we earn on qualifying purchases.
The proposal is an individual Internet-Draft connected to the IETF’s AI Preferences working group. An Internet-Draft is a proposal for discussion, not an adopted internet standard. At a working group meeting on October 21, one of the group’s chairs said, “We haven’t declared consensus on anything yet.” The draft itself has no formal standing in the IETF standards process.
The companies’ announcements refer to the same draft, but their commitments are not identical. Google’s stated approach is to treat the shared training signal like a disallow rule for Google-Extended when a site has no separate Google-Extended rule. OpenAI describes its commitment as a voluntary effort and says its existing crawler controls will continue to apply. Anthropic’s announcement says implementation has begun, though its crawler help page on October 31 still described its existing per-crawler controls.
Meta communications chief Andy Stone expressed support for publishers’ ability to state preferences and called a common signal an important step, but did not say whether Meta would honor it. The source material reviewed also reports no commitment from Apple, Amazon, xAI or Perplexity. Support from four companies therefore does not establish participation across the AI industry.
“Anthropic “has already started implementing support for this draft.””
— Anthropic standards lead Kevin Kelley
Participation and Enforcement Remain Open
Several points remain unsettled. The signal is voluntary, and the announcements do not establish how compliance will be monitored or what recourse a publisher would have if a company disregarded the preference. The draft’s status could also change through the IETF process, but there is no formal standards adoption in the information available here.
Coverage across the industry is uncertain: Meta has expressed support without making an implementation commitment, and the reviewed source material contains no commitments from several other named AI companies. The exact dates and scope of support also differ by provider. Anthropic’s implementation across its services is still in progress; Google and OpenAI have described intended timelines rather than a fully completed rollout.
It is also not clear from these announcements how every company will handle edge cases, such as content copied or supplied by third parties, or whether the signal will affect AI systems beyond the training use it names. The shared rule does not itself answer those questions.
Check Rules Before Each Rollout
Website operators who want to use the signal can add the rule group to robots.txt and review it alongside their current crawler instructions. The shared rule is not a replacement for company-specific controls during the rollout: Anthropic, OpenAI and Google each describe existing controls that continue to matter, and the companies’ implementation dates differ.
Microsoft’s timeline requires a separate review. Its new training and grounding preferences begin on February 1, 2027, as Microsoft crawls sites; NoArchive and NoCache stop counting as training or grounding preferences after March 1, 2027. Publishers using those tags should consult Microsoft’s documentation and decide separately whether they want to express a preference about AI grounding.
The next developments to watch are each provider’s published implementation details, any change to Meta’s position, and progress of the Internet-Draft in the IETF working group. Until those are clearer, the rule is best understood as a shared request that some major providers have agreed to honor—not a universal or technically enforced ban.
Key Questions
What line should a website add to opt out of AI training?
Add a robots.txt rule group with User-Agent: __usage_ai-training and Disallow: /. The signal expresses a preference against using the site’s content for generative AI training.
Does the rule stop AI companies from crawling a website?
No. The source describes the signal as a request, not a technical barrier. It does not itself block crawling or guarantee that every company will follow it.
The shared signal is for model training. Microsoft announced a separate page-level preference for content used to ground answers in Bing, Copilot and Microsoft 365; the two controls address different uses.
When will the companies honor the signal?
Timelines vary. Anthropic says implementation has started, OpenAI aims for the coming months, Google aims for 2027 and lists early 2027 in its documentation, and Microsoft sets February 1, 2027, for its support to begin as it crawls.
Is the signal an official internet standard?
No. It is an individual Internet-Draft connected to the IETF AI Preferences working group and has not been adopted as an official standard.
Source: ThorstenMeyerAI.com
