Anthropic has disclosed that its AI models have reached a critical internal safety threshold, raising concerns about the company’s ability to maintain human oversight as it uses AI to develop more advanced versions of its Claude models.

The safety milestone, included in Anthropic’s November 10 amended IPO filing, is that advanced AI models are now contributing enough to their own development that the pace of innovation could potentially double, according to the company’s assessment.

However, Anthropic emphasized that its determination is based partly on projected advances in future models, rather than the demonstrated capabilities of existing systems. This is an essential point because Anthropic has not established that its current models can independently produce successive generations of advanced AI.

Still, the company is preparing additional safety measures as it evaluates the implications of faster AI development. Anthropic added the disclosure comes as it prepares for a stock market debut that could value it at approximately $1.4 trillion.

AI Research Acceleration Raises Concerns

Anthropic’s Responsible Scaling Policy establishes safety requirements designed to address the risks posed by more capable AI systems. One of the policy’s key benchmarks concerns automated research, especially the potential for AI models to accelerate the development of their next versions.

The threshold can be reached when research automation is expected to approximately double the pace of AI capability improvements, compared with progress achieved without substantial AI assistance. This speed of self-improvement could create a feedback loop in which more capable models help researchers build still more powerful systems, reducing the time available to evaluate potential risks.

Anthropic already uses Claude to support software development and AI model training. These capabilities are core to the company’s strategy for improving performance and reducing research costs. But the company has acknowledged that relying more heavily on AI could make some development processes difficult for human researchers to fully understand.

Crossing the threshold does not automatically require the company to suspend research or stop deploying its models.

Independent Research Finds Faster Progress

Independent AI evaluation organization METR has reported evidence supporting concerns about the pace of Anthropic’s development. The nonprofit estimated that progress on relevant AI benchmarks has accelerated by a factor of approximately 1.9 since October 2025. However, this finding doesn’t independently establish that Anthropic has satisfied every requirement of its internal safety threshold.

METR also questioned an earlier Anthropic assessment that placed the company below the critical benchmark in August. The organization considered it possible that the threshold had already been reached by mid-July.

These findings reveal the difficulty of measuring AI research acceleration, particularly when companies rely on internal systems and unreleased models.

Anthropic’s safety framework also considers a more advanced scenario in which AI systems could perform the work of its entire research and engineering workforce at competitive costs. The company has not claimed to have reached that level of automation.

Anthropic’s safety disclosure arrives during a major fundraising effort involving several leading tech companies. The company has established an IPO price range of $70 to $78 per share, with trading potentially beginning November 19.

Google and NVIDIA are each committing $10 billion to Anthropic shares as part of a $26 billion private placement involving Microsoft, AMD, Broadcom, Micron and TSMC.