Anthropic has quietly pitched US lawmakers on draft legislation that would give the federal government limited authority to block the release or use of advanced artificial intelligence models deemed to pose imminent risks of catastrophic harm, reopening debate over how Washington should regulate the most powerful AI systems.
The proposal would allow the government to seek restrictions on a model only when it presents a foreseeable and imminent risk of causing more than $1 billion in damages, or killing or seriously injuring more than 50 people, according to a Politico report on Friday, which cited three people familiar with the effort.
The risks covered by the draft bill would have to arise from the creation or release of a biological weapon, an AI model evading its developer’s controls, or a cyberattack. The legislation would also require developers of frontier AI models to conduct safety testing, disclose findings and submit their systems to independent evaluation.
Register for the next Tekedia Mini-MBA.
Register for Tekedia AI in Business Masterclass.
Join Tekedia Capital Syndicate and co-invest in great global startups.
An Anthropic spokesperson told Reuters that the bill represents proposals contained in the company’s Advanced AI Framework, released earlier this year. The framework forms part of Anthropic’s push for formal safeguards as AI models become more capable and potentially more consequential in areas such as cybersecurity and biological research.
The proposed legislation would give the government the “ability to block or deter dangerous deployments of the most powerful models,” the spokesperson said.
The Department of Commerce, which would have a central role in the proposed oversight system, did not immediately respond to Reuters’ request for comment.
Under the draft legislation, independent evaluators would register with the Commerce Department and periodically assess AI developers and their frontier models, according to Politico. Although AI companies could pay for these assessments, compensation could not be tied to the outcome of an evaluation.
The arrangement is intended to establish an external assessment process without necessarily separating safety oversight entirely from the companies developing the technology. Its effectiveness would depend in part on how evaluators are selected, how their independence is enforced, and whether they can identify risks that developers’ internal testing may overlook.
The draft also establishes a reporting requirement for serious findings. If evaluators determine that a company’s frontier models present an “imminent enumerated risk,” they would have to notify the Commerce Department within 72 hours of completing their assessment.
That provision would create a formal route for escalating concerns about potentially dangerous AI deployments. It could also place pressure on developers and evaluators to establish clear standards for determining when a risk is sufficiently immediate and severe to trigger government intervention.
However, the legislation’s proposed definition of actionable harm is narrow. The government would need to establish a foreseeable and imminent threat meeting the specified thresholds, rather than relying solely on a general concern that a model might eventually be misused or cause harm.
This distinction could become significant as AI systems grow more capable. Some risks may emerge gradually, through increasingly sophisticated cyber operations or the accumulation of capabilities that are difficult to evaluate before deployment. Others may depend on a chain of events involving a model, external tools, and human decisions.
The draft’s emphasis on imminent, measurable harm could make intervention more legally constrained, while leaving questions about how authorities should respond to serious risks that do not fit neatly within the specified categories.
Draft Seeks to Limit Government Power Over AI Development
One of the proposal’s most consequential features is its restriction on which parts of the federal government could seek to halt an AI model.
According to Politico, the Commerce Department could ask a judge to suspend a model after determining that it presents the relevant risk. A federal judge in the US District Court for the District of Columbia could then order the suspension.
The draft explicitly prohibits other federal departments, agencies, or officers from independently suspending the development, deployment, or internal use of a model, the report said.
This would concentrate intervention authority within a defined process rather than allowing multiple agencies to impose separate restrictions. It could provide AI developers with greater clarity about the circumstances in which their models might be blocked, while establishing judicial review as a safeguard against potentially broad or arbitrary government action.
At the same time, the proposed limits could complicate responses to risks that span several areas of national security and public safety. Cybersecurity, biological threats and other potential consequences of advanced AI may involve agencies with different responsibilities and technical expertise. Restricting independent suspension powers could reduce overlapping regulatory decisions, but it would also make the coordination between Commerce officials, evaluators and the courts especially important.
Experts expect the draft’s practical impact to be spurred not only by the risk thresholds but also by how quickly regulators could investigate an alert, establish that the statutory conditions had been met, and obtain judicial approval.
The proposal remains a draft that Anthropic has presented to lawmakers, rather than an enacted federal requirement. Its eventual form, political support, and legislative prospects have not been established by the reported details.
AI Safety Debate Shifts Towards Enforceable Rules
Anthropic’s initiative comes as policymakers face growing pressure to determine how far government oversight should extend over increasingly powerful AI systems. Voluntary safety commitments can encourage testing and disclosure, but they may offer limited assurance if companies face commercial incentives to release models quickly or if developers disagree about acceptable levels of risk.
The proposed bill attempts to bridge that gap by combining mandatory testing and independent evaluation with a narrowly defined mechanism for preventing deployments that threaten severe harm.
Its approach also raises a central regulatory question about how to preserve room for innovation while ensuring that a model capable of enabling major cyberattacks or biological threats cannot be released without meaningful scrutiny.
The $1 billion damage threshold and the benchmark of more than 50 deaths or serious injuries would provide specific reference points for government intervention. But thresholds alone cannot resolve questions about how risks should be measured, how much evidence is required, or how authorities should account for uncertainty when evaluating a model’s potential capabilities.
The independent evaluation system would consequently be critical. Assessors would need sufficient access to models and relevant technical information to identify dangerous capabilities, while remaining independent enough to report findings that could delay or prevent a commercially valuable release.
Allowing developers to pay evaluators without tying compensation to the outcome is one proposed safeguard, although the draft’s details do not by themselves establish how conflicts of interest would be prevented or investigated.