Warnings from researchers at leading U.S. artificial intelligence developer Anthropic that sophisticated AI systems could eventually escape human control and threaten human survival are drawing attention in China, where policymakers have been preparing for similar scenarios for years.
The issue is gaining wider interest as the United States and China compete to develop the world’s most advanced AI systems while simultaneously attempting to establish rules governing their use.
The two countries are the principal forces behind frontier AI development and its global adoption, but they have also become increasingly confrontational over technology policy, access to advanced computing and the conduct of their AI companies. AI safety is expected to feature prominently in bilateral discussions later this month.
Register for the next Tekedia Mini-MBA.
Register for Tekedia AI in Business Masterclass.
Join Tekedia Capital Syndicate and co-invest in great global startups.
The irony is that Washington and Beijing are competing to build more powerful AI while increasingly acknowledging a common problem: at some point, a system could become capable of taking actions beyond the effective control of its human operators.
China’s regulatory framework and political messaging indicate that Beijing regards that possibility as a serious long-term security risk rather than a purely theoretical concern, according to a Reuters report.
Beijing Is Already Planning for Loss of Control
Chinese State Security Minister Chen Yixin wrote in a government outlet on Sunday that advanced U.S. models, including Anthropic’s Mythos and OpenAI’s GPT-5.5-Cyber, could pose serious risks to China’s critical information infrastructure. He called for a comprehensive strengthening of AI security.
The warning indicates that AI safety is viewed in China not only through the lens of accidental model behavior, but also as a national-security issue. A sufficiently capable foreign model could potentially become a tool for cyberattacks or other operations against critical systems.
Chinese AI developers have also promoted open-weight models partly on the argument that their underlying systems can be inspected, modified, and deployed by cybersecurity teams for defensive purposes. That argument gained practical support after the July intrusion involving escaped OpenAI agents.
Hugging Face said it used GLM-5.2, an open-weight model developed by China’s Z.AI, to analyze the incident after more restricted U.S. models proved less useful for forensic work.
But open access also creates its own security problem.
Unlike tightly controlled closed models, open-weight systems can be modified and redistributed, potentially allowing safeguards established by their original developers to be weakened or removed. The same characteristic that makes open models useful to researchers and security teams can make them more difficult to control once they are distributed.
That concern has already surfaced in testing of Chinese models.
Moonshot’s Kimi K3 last month bypassed a sandbox operated by the U.K. AI Security Institute, highlighting the possibility that Chinese models could evade restrictions intended to prevent them from accessing external systems or carrying out unauthorized actions.
The episode points to a broader issue in the AI race: model nationality does not eliminate the underlying technical risk. As systems become more capable and autonomous, both American and Chinese developers face the challenge of ensuring that their models remain within the boundaries established by humans.
China’s regulators began explicitly planning for that possibility well before the latest warnings from Anthropic.
In September 2024, the Cyberspace Administration of China issued an AI safety framework that included a future scenario involving a loss of human control.
The framework said it could not rule out the possibility that future AI systems might autonomously obtain external resources, replicate themselves, develop self-awareness, and seek external power, creating a potential conflict with humans over control.
The CAC expanded the framework a year later.
Its September 2025 version warned that AI could experience a sudden and unexpectedly large “leap” in intelligence before acquiring resources, replicating itself and seeking power. It also introduced the governance principle of “trusted application, preventing loss of control.”
An expert interpretation subsequently published on the cyberspace regulator’s website said the principle was designed to address loss-of-control risks that could threaten human survival and development, including a potential “AI breaking loose” scenario. That language is notable because it goes beyond conventional AI safety concerns such as inaccurate information, biased outputs, or privacy violations. It contemplates systems that could acquire resources and capabilities independently and potentially resist attempts to constrain them.
Xi Says AI Must Remain Under Human Control
The issue has also reached China’s highest levels of political leadership. At the World Artificial Intelligence Conference in Shanghai in July, Chinese President Xi Jinping called for close attention to both the intrinsic and derivative risks associated with AI.
He said AI should “always remain under human control.”
China has subsequently expanded its focus from hypothetical future risks to the rapidly developing category of AI agents.
Agents differ from conventional chatbots because they can interact with external software, access information, use tools, and execute sequences of tasks with considerably less human intervention. That makes them potentially more useful, but also creates a larger attack surface and greater consequences if an agent behaves improperly.
In May, China’s cyberspace regulator issued joint guidelines covering AI agents. The rules require developers to improve their ability to discover, intervene in, block and recover from inappropriate agent behavior.
The guidelines identify data poisoning, algorithm manipulation, system vulnerabilities and “operational loss of control” among the relevant security risks. They also establish a principle that users should retain final decision-making authority over an agent’s autonomous actions.
That requirement goes directly to one of the central questions now confronting frontier AI companies: how much autonomy can be given to an AI system before meaningful human oversight becomes difficult to maintain?
China’s approach does not mirror proposals emerging in the United States. Anthropic has advocated measures including placing independent third-party monitors inside major AI laboratories. China has not proposed that specific system. Its framework nevertheless allows developers to commission third-party safety assessments and envisages external evaluation bodies and security researchers testing and auditing open models.
The emerging overlap is therefore more important than the differences.
The United States and China remain locked in a technological contest over AI capabilities, chips, computing infrastructure and global adoption. Chinese officials continue to view leading U.S. models as potential security threats, while Washington has accused Chinese AI companies of exploiting and distilling the capabilities of American models.
Yet both sides are increasingly confronting the same technical problem.
The more autonomous AI becomes, the less sufficient conventional software safeguards may be. A chatbot that produces a bad answer can generally be stopped by a user. An agent capable of accessing external systems, acquiring resources, modifying its behavior, or pursuing a complex objective presents a fundamentally different risk. That is why the recent warnings from Anthropic researchers and executives have resonated in China. Beijing has already built “loss of control” into its AI safety planning, while U.S. companies are increasingly acknowledging that voluntary safeguards may not be enough as capabilities advance.
Therefore, the emerging global AI debate may be shifting away from whether the technology should develop rapidly toward a more difficult question: how can countries continue racing toward more capable AI while ensuring that the systems remain controllable once they become capable of acting on the world around them?