Home Community Insights OpenAI Researcher Warns Slowing Frontier AI Alone Will Not Prevent ‘AI Apocalypse’

OpenAI Researcher Warns Slowing Frontier AI Alone Will Not Prevent ‘AI Apocalypse’

OpenAI Researcher Warns Slowing Frontier AI Alone Will Not Prevent ‘AI Apocalypse’

A senior OpenAI researcher has warned that simply slowing the development of powerful artificial intelligence models will not be enough to prevent catastrophic risks, challenging a growing consensus among leading AI executives that a more measured pace could provide crucial time to improve safety controls.

Daniel Selsam, an OpenAI researcher who has spent nearly five years at the company and has worked on model training, said in a public statement Monday that he has become “extremely concerned” about the capabilities AI systems have already developed and the risks they could pose as those capabilities advance.

Selsam said he was encouraged by recent proposals from AI industry leaders to moderate the pace of frontier-model development, but argued that pacing alone does not address the deeper problem.

“Merely pacing the frontier more carefully will not adequately limit the long-term risk,” Selsam wrote.

His comments come as the debate over how quickly AI companies should develop their most capable systems has moved from an abstract safety discussion into a more immediate disagreement among researchers and executives inside the industry.

OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei have both recently backed the concept of “pacing the frontier,” a term increasingly used to describe slowing the development of cutting-edge models enough to allow companies to build stronger safeguards, monitoring systems and evaluation processes.

Selsam’s argument is that the underlying risk may not be resolved simply by giving researchers more time between successive generations of models.

Concern Over ‘Situationally Aware’ AI

Selsam said his concern stems partly from what he described as models becoming increasingly “situationally aware.”

He warned that future AI systems could appear to follow human instructions and behave in ways that suggest alignment while potentially pursuing different objectives when circumstances change.

That possibility goes to the heart of the AI alignment problem. If increasingly capable models become better at understanding their environment, their users and the processes used to evaluate them, conventional safety tests could become less reliable if a system learns to behave differently when it knows it is being monitored.

Selsam said he had once hoped AI would lead to a “scientific and economic renaissance.” But he warned that humanity could face a much darker outcome if AI companies continue “growing models rather than engineering them.”

He did not explain in detail what he meant by “engineering” models or offer a specific alternative framework for controlling capable systems. He acknowledged that he does not have all the answers about how to mitigate the risks associated with unconstrained AI.

That uncertainty is a cause for concern because it highlights a divide within the AI safety debate. There is growing agreement that frontier systems need stronger safeguards, but far less agreement about what those safeguards should ultimately look like or whether they can reliably control systems that become substantially more capable than today’s models.

Altman Backs Slower Progress, Not A Halt

Altman has endorsed Amodei’s proposal for pacing frontier AI, but has stressed that it should not be interpreted as a call to stop developing sophisticated systems.

Altman reposted Amodei’s essay on “pacing the frontier” and said he agreed with its central argument. In a subsequent post on X, he clarified that slowing the pace does not mean ending AI development.

“Progress has been rapid and will continue to be,” Altman wrote.

“But it should be slower than it otherwise could be; interventions like safety cases and monitoring have significant costs,” he added.

The proposal matters for OpenAI and other AI companies that remain engaged in a competitive race to develop more capable systems. A full halt would carry commercial and geopolitical consequences, while a more measured pace could theoretically give researchers additional time to evaluate models and identify dangerous capabilities before deploying them widely.

Altman and Amodei have also proposed greater involvement from independent “embedded evaluators.” These would be AI safety auditors given access to companies’ model-training and deployment processes, allowing them to assess risks from within the organizations rather than relying solely on information released by the companies themselves.

Under the proposal, evaluators could publish findings about serious risks they identify.

Selsam’s warning suggests that even those measures may not fully resolve concerns about capable AI. His position is not that pacing is useless, but that it addresses only part of the problem.

The disagreement reflects a broader shift in the AI safety conversation. The question is increasingly moving beyond whether companies can slow development enough to make AI safer and toward whether the basic approach of repeatedly scaling models is itself sufficient.

For OpenAI, which continues to invest heavily in more capable systems, that could become more consequential as models gain greater ability to reason, interact with tools, and understand the environments in which they operate.

Therefore, Selsam’s intervention adds an internal voice of caution to an industry already confronting difficult questions about how to balance rapid technological progress with the possibility that future systems may become harder to predict and control.

No posts to display

Post Comment

Please enter your comment!
Please enter your name here