Anthropic CEO Dario Amodei has called on artificial intelligence companies to slow the pace at which they improve increasingly capable models, warning that the industry needs more time to strengthen safety systems before AI reaches capabilities that could create serious real-world risks.
In an essay published Saturday, Amodei proposed a three-step framework for “pacing” AI development without sacrificing commercial competitiveness or the United States’ lead in the technology.
“But over the last few months, I have become convinced that fully addressing the risks requires even more prudence — not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up,” he said.
Register for the next Tekedia Mini-MBA.
Register for Tekedia AI in Business Masterclass.
Join Tekedia Capital Syndicate and co-invest in great global startups.
“We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain. Two things have convinced me.”
The proposal comes as AI companies race to build more capable systems and as concerns grow within the industry that advances in reasoning, autonomy and cyber capabilities may be outpacing the safeguards designed to control them.
“But like many technologies before it, AI brings risks, and because it is such a powerful technology, these risks are serious. I’ve written a lot about them too. They include the risk of losing control of AI systems, misuse of AI for cyberattacks and bioterrorism, and serious economic disruption. A race to the bottom, spurred by commercial incentives, can make these risks more acute,” he said.
Amodei stressed that pacing does not mean stopping AI research or halting model training.
“To be clear, pacing does not mean halting model training or technical progress,” he wrote.
Instead, he said companies should ensure they have adequate time to align and safeguard capable systems and allow independent evaluators to verify that those protections work.
Anthropic has already committed unilaterally to the first part of Amodei’s proposal. Under it, independent third-party evaluators would receive employee-level access to the company, allowing them to examine safety practices and report incidents.
The second step calls for leading AI companies in democratic countries to coordinate on common safety standards. The third would extend that coordination to democratic and authoritarian governments.
The proposal has gained wide interest because it seeks to address the competitive problem at the heart of AI safety. A unilateral slowdown could leave one company at a disadvantage if competitors continue developing more powerful systems. Amodei’s approach instead calls for companies and governments to establish shared standards that could make restraint less commercially costly.
AI Safety Concerns Are Becoming Harder To Dismiss
Amodei’s essay followed a public dispute over the risks posed by powerful AI systems.
Jacob Coxon, a researcher who recently left Anthropic and previously worked at OpenAI, said this week that he resigned because he believed the companies were taking unacceptable risks. Coxon said people developing AI “earnestly believe that it could kill us all by the end of the decade.”
Such warnings remain controversial, but concerns about catastrophic AI risks are not new. In 2023, Amodei, OpenAI CEO Sam Altman, and other prominent AI researchers and executives signed a statement arguing that reducing the risk of extinction from AI should be treated as a global priority alongside threats such as pandemics and nuclear war.
Amodei now believes that circumstances have changed since that earlier debate.
He said slowing or pausing development made little sense in 2023 because AI systems were not yet powerful enough to take meaningful action in the real world and lacked capabilities involving significant deception, manipulation, cheating or cyberattacks.
The implication of his latest argument is that the question is no longer simply whether AI could eventually become dangerous. It is whether the industry’s safety infrastructure is developing quickly enough alongside the models themselves.
“I continue to believe that AI can enormously improve the quality of human life,” Amodei wrote.
He added that those benefits depend on building the technology correctly and using additional time to improve safeguards.
His proposed approach therefore stops short of the broad moratoriums that have occasionally been advocated by AI safety campaigners. Instead, it seeks a temporary reduction in the speed of capability development to create more time for evaluation, monitoring and alignment.
Altman And Musk Endorse The Idea
Amodei’s proposal received support from two of the most prominent figures in the AI industry, OpenAI CEO Sam Altman and Elon Musk.
Altman said on X that he agreed the industry needs to pace the development of advanced AI capabilities, adding that the issue had become a “primary topic” of discussion at OpenAI in recent weeks.
“Committing to having independent evaluators with employee-like access is a great idea, and we will do the same,” Altman wrote. “We’ll have more to share soon.”
The endorsement is significant because OpenAI is one of Anthropic’s closest competitors in developing frontier AI systems. If other leading companies adopt similar independent evaluation arrangements, Amodei’s first proposed safeguard could become an emerging industry norm rather than an Anthropic-specific policy.
OpenAI’s chief scientist, Jakub Pachocki, had already raised similar concerns earlier this month. He warned that no AI company had “solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer.”
Pachocki said he expected and hoped voluntary slowdowns would become commonplace until companies established shared safety thresholds.
Musk also endorsed Amodei’s proposal, writing simply: “Dario is right.”
The support is particularly striking given Musk’s previous criticism of Anthropic. He had accused the company of hostility toward Western civilization and suggested it was destined to become “misanthropic.” His stance softened after Anthropic announced a major computing deal with SpaceX in May.
Musk said at the time that the people he met at Anthropic were highly competent and appeared deeply concerned with doing the right thing.
Amodei’s proposal now puts the industry in a more complicated position. AI companies are competing aggressively over model capability, customers and computing resources, yet some of their leaders are simultaneously acknowledging that moving at maximum speed may no longer be compatible with responsible development.
Amodei believes even a relatively short delay could matter. If slowing development bought the industry another year or two before models reached what he called “critical levels of capability,” he argued, that time could be used to improve alignment and substantially reduce the probability of a serious failure.
That makes his proposal less a call to stop the AI race than an attempt to create a safety interval within it. However, there is concern about companies collectively accepting that interval without fearing that rivals will use the extra speed to gain an advantage.