Salesforce CEO Marc Benioff has urged artificial intelligence companies to act responsibly as powerful models become embedded in businesses and everyday life, warning the industry not to repeat the mistakes made during the rise of social media.

Benioff’s comments came as the technology industry confronts a growing debate over AI safety, intensified by Anthropic CEO Dario Amodei’s call for frontier AI developers to slow the pace of model development so that safety measures can keep up with rapidly advancing capabilities.

Speaking to CNBC’s Jim Cramer during Salesforce’s annual Dreamforce conference in San Francisco, Benioff said the social media industry provided a cautionary example of what can happen when powerful technology expands faster than safeguards and accountability.

“A lot of companies got hurt, a lot of individuals got hurt through social media,” Benioff said. “We don’t want that to happen in AI.”

Benioff has spent years criticizing the social media industry’s impact on society. In 2018, he described Facebook as “the new cigarettes” and called for greater government regulation of social media companies.

His warning about AI is notable because Salesforce is closely connected to the companies developing the technology. The enterprise software giant has increasingly incorporated generative AI into its products and partnerships, putting it directly in the middle of the commercial expansion of the technology.

Last month, Salesforce unveiled Claudeforce, which includes a plugin allowing customers to use Anthropic’s Claude to access customer information stored in Salesforce and perform tasks including composing emails and updating records.

Amodei appeared alongside Benioff at Dreamforce on Tuesday, reinforcing his argument that AI developers should take greater responsibility for the pace and safety of their technology.

“I think that’s the way to lead the industry forward, to set an example, to say that everyone can always be better,” Amodei said.

Benioff Stops Short of Calling for a Slowdown

While Benioff emphasized responsibility and ethics, he stopped short of directly criticizing AI developers or endorsing Amodei’s call for a slowdown.

Instead, he framed the issue as one of corporate responsibility. AI companies, he argued, need to consider how their products affect customers, communities, and society as the technology becomes more capable.

“We want AI to take care of these things and be held responsible, and take care of these actions and be ethical,” Benioff said. “The heart of ethics is responsibility.”

He also appealed directly to technology companies developing frontier AI systems.

“We really look at a light on our brethren here in San Francisco and all over the world and say, ‘We want you to be ethical in your actions and how you take care of your communities and the world,’ because this technology is very powerful,” he said.

The comments add another perspective to an increasingly public disagreement among technology executives over how to manage AI’s rapid development.

Amodei has noted that the industry should deliberately reduce the pace of capability improvements. OpenAI CEO Sam Altman has backed the need for greater coordination and monitoring, while Meta CEO Mark Zuckerberg and Nvidia CEO Jensen Huang have emphasized the ability of companies to build safety and alignment into AI systems while continuing development.

Benioff’s position is closer to a responsibility-focused approach. His argument does not require companies to halt development, but it places greater emphasis on how they deploy the technology and the obligations that come with controlling increasingly powerful systems.

His idea matters for Salesforce, whose business depends on persuading enterprises to place AI inside core business processes. As AI agents move from generating content to accessing company databases, updating records and taking actions on behalf of employees, questions around reliability, security and accountability become commercial issues as well as questions of ethics.

The debate is also unfolding against a changing view of AI’s impact on enterprise software. Salesforce and other software companies were hit earlier this year by concerns that sophisticated AI models could disrupt traditional applications by allowing businesses to accomplish tasks without relying as heavily on conventional software.

Those concerns have eased in recent months as investors have increasingly considered the possibility that AI could become an additional layer of enterprise software rather than simply replace it.

Salesforce shares rose more than 4.5% in the previous session as enterprise software stocks rallied following Amodei’s call for a slower pace of AI development. The shares gave back some of those gains on Tuesday.

Salesforce remains more than 50% above its 52-week low of about $146 reached in late June.

For Benioff, however, the major issue is whether the companies building and deploying the technology can accept responsibility for what happens as its capabilities expand.

His warning draws a direct line from the social media era to the AI boom: technologies can create enormous commercial value while also producing consequences that become harder to manage once adoption reaches a massive scale.

“We don’t want that to happen in AI,” Benioff said.

Nvidia CEO Jensen Huang has rejected calls for new laws to govern artificial intelligence, noting that AI remains a technology built by humans and can therefore be controlled through engineering, existing laws and market forces.

Speaking at Salesforce’s Dreamforce conference on Tuesday, Huang pushed back against descriptions of AI as an emerging form of “alien mind,” saying the technology is ultimately software and computing systems designed by people.

“Safety is an engineering problem, not a legal one,” Huang said. “We’re developing software after all. We’re developing computing systems after all. It’s a complicated computing system, but it’s ultimately a computing system.”

His position puts him on the more permissive side of a growing, important debate over how governments and the technology industry should manage the risks created by sophisticated AI systems. While some AI researchers and executives have argued that frontier models may require new forms of oversight, Huang sees existing legal frameworks and market incentives as sufficient to push companies toward safer products.

“If we’re not confident about the safety of the products, like all companies, like you and I, all the companies here, if you build a product or a service, and you’re not confident in its functionality, capability, or safety, then don’t release it. And so that’s a very obvious thing to do,” he said.

Huang argued that companies can slow development when necessary without sacrificing the speed of technological progress.

“You pace yourself until you are confident you’re releasing something that the market would appreciate,” he said. “The market forces are already there. We don’t need any new laws. We don’t need new regulations.”

He also rejected the idea that companies must choose between rapid innovation and safety.

“I think innovation, speed, and safe products … it’s a false choice,” Huang said. “You could definitely have both at the same time.”

For Huang, the basic principle is that companies should move as quickly as they can while stopping when they believe a product is not ready or safe enough to release.

The Case for Market Discipline

Huang’s argument rests on a familiar model from the technology industry. Companies have commercial incentives to avoid releasing products that fail, cause damage, or expose customers to unacceptable risks. Existing liability laws can also impose financial consequences when products cause harm.

The approach is notable coming from Huang, whose company sits at the center of the current AI boom. Nvidia supplies the processors and computing infrastructure that power many of the world’s most advanced AI systems and has benefited enormously from the rapid expansion of AI development.

Huang has also become increasingly vocal about AI’s economic potential. “I’m more ambitious than ever,” he said. “As a result of our ambition, and with the product productivity boost that we get from AI, the sky’s the limit for us. The sky’s the limit for our company. The sky’s the limit for every industry, for every single country.”

That commercial exposure makes his preference for allowing AI development to proceed with limited new regulation unsurprising in the context of Nvidia’s business interests, even though the underlying argument does not depend on Nvidia’s position.

However, there is concern about the market’s ability to reliably identify and punish unsafe AI products before significant damage occurs.

Software companies have long released products with unintended consequences even when they were not deliberately designed to cause harm. The 2024 CrowdStrike software failure, for example, disrupted airlines and businesses around the world after a faulty update caused widespread computer crashes.

AI introduces additional complications because the behavior of advanced models can depend on how they are prompted and deployed, and because their outputs can change as systems become more capable.

AI-related incidents have already raised questions about those risks. OpenAI has faced scrutiny over the behavior of its models, including an incident involving a model hacking into Hugging Face. The company has also faced lawsuits concerning the alleged effects of prolonged interactions between young people and its chatbot.

Those cases do not establish that AI companies deliberately released unsafe products. They do, however, demonstrate why relying solely on a company’s own judgment about whether a system is ready for release can be contentious.

Regulation Versus Self-Regulation

Huang’s position also leaves open a question that sits between government regulation and unrestricted development: whether the industry can establish credible standards for itself.

Industry self-regulation could allow AI companies to develop common safety practices without imposing a comprehensive regulatory framework on the technology. It could include independent testing, disclosure requirements, model evaluations, and agreed thresholds for deploying particularly capable systems.

The challenge is coordination. AI development is global and highly competitive, meaning companies that voluntarily impose additional constraints could worry that competitors will use the opportunity to move faster.

That concern becomes more complicated when the competition extends across national borders.

Microsoft CEO Satya Nadella raised that issue at the All-In Summit on Monday, saying that Chinese AI companies should have an interest in addressing the same safety problems as their US counterparts.

“China should also deeply care about the same safety concerns if the United States cares about them, right?” Nadella said. “Why should it be different for them?”

His argument points to a problem that cannot easily be solved by domestic regulation alone. If safety standards vary significantly between countries, companies operating under stricter rules could face competitive pressure from firms operating under weaker ones.

For Huang, however, the priority remains technological progress combined with engineering discipline rather than additional legislation.

His emphasis on open-weight models and competition among AI developers also fits into a broader argument that market competition can provide a counterweight to concentrated control by a small number of proprietary AI laboratories.

The debate is therefore about where responsibility should sit: with engineers designing and testing models, companies deciding when products are ready, markets rewarding or punishing failures, governments establishing legal obligations, or some combination of all four.

For now, Huang is clearly arguing for the first three without adding a new layer of AI-specific regulation.

That position carries particular weight because of Nvidia’s influence over the infrastructure underlying the AI industry and Huang’s growing influence in Washington. He demonstrated that influence again this week by showing that he has direct access to President Donald Trump.

So far, it is not clear whether existing laws and market incentives can keep pace with sophisticated AI systems. Huang’s argument is that the technology should be treated as an engineering challenge and governed accordingly. The counterargument is that the consequences of an AI failure may sometimes emerge faster than courts, regulators, or markets can respond.

Elon Musk is calling on the world’s leading artificial intelligence companies to subject their models to independent testing by competitors before releasing them to the public, proposing a form of industry peer review as concerns grow over the safety of sophisticated AI systems.

Speaking at the All-In Summit in Los Angeles on Monday, Musk said SpaceX’s xAI, OpenAI, Anthropic, Google, Meta and “three or four of the leading Chinese companies” should allow rival developers to run a common “test harness” against their models and identify potential safety problems before deployment.

“So, you know, instead of grading your own homework, you would at least have competitors grading your homework and raising the alarm if they see concerns,” Musk said.

The proposal comes as the AI industry faces a more intense debate over how quickly frontier models should be developed and whether voluntary safeguards are sufficient. Leaders of Anthropic, OpenAI and other AI companies have recently warned about the risks posed by powerful systems and called for a slower pace of development.

Musk and OpenAI CEO Sam Altman were among the technology executives who backed Anthropic CEO Dario Amodei’s proposal for slowing frontier AI development, creating an unusual degree of agreement among rivals that have otherwise been engaged in an aggressive race for users, computing capacity and market share.

Musk’s latest proposal shifts that discussion toward a specific mechanism: forcing AI developers to expose their systems to scrutiny from companies with competing commercial interests.

“The odds that you will find issues are dramatically greater,” Musk said, while acknowledging that the peer-review approach would not be a perfect solution.

The idea broadly resembles Amodei’s proposal for “embedded evaluators,” in which independent third parties would be given sufficient access to assess the safety of frontier models and verify that companies are following their commitments. Musk’s proposal goes a step further by explicitly bringing competing AI developers into the testing process.

AI Safety Debate Collides With Commercial Competition

The renewed safety debate was partly triggered by warnings from AI researchers about the possibility that future systems could pose catastrophic risks.

Jacob Coxon, a former researcher at Anthropic who had also worked at OpenAI, announced that he had left Anthropic and accused leading AI laboratories of “gambling with our lives.” His comments were followed by a warning from Evan Hubinger, an alignment lead at Anthropic, who said he agreed with Coxon and personally estimated that AI could kill all humans with a probability of more than 10% within the next decade.

Those warnings have added urgency to a debate that had largely centered on whether AI regulation should be imposed by governments or developed voluntarily by the companies building the technology.

The Trump administration has pushed back against calls for broader restrictions. President Donald Trump posted on Truth Social on Monday describing fears about AI as a “hoax” and a “scam.”

National Economic Council Director Kevin Hassett told CNBC on Tuesday that the private sector is the “right place” to address concerns surrounding AI. He said the government was monitoring the industry and would use “law enforcement when necessary to make sure that the firms are acting responsibly.”

The disagreement exposes a major tension in AI governance. The companies developing frontier models argue that they are best positioned to understand and manage rapidly evolving technical risks, while policymakers and researchers have raised questions about whether firms facing intense competitive pressure can be relied upon to police themselves.

Musk’s proposal is an attempt to address part of that problem without placing the testing mechanism entirely in government hands. If OpenAI, Anthropic, Google, Meta, xAI and major Chinese developers were required to test one another’s models, companies would have an incentive to search for vulnerabilities that their competitors might otherwise overlook.

But the commercial incentives are complicated.

Musk acknowledged that the other AI companies competing with his businesses have not agreed to his proposal. Each company has reasons to protect proprietary model information, while giving competitors access to sophisticated testing environments could reveal weaknesses, capabilities, or other information with commercial value.

The proposal also raises questions about who would control the testing framework, what constitutes a safety failure, and whether companies would be required to disclose problems discovered in a rival’s system.

Musk said the mechanism should be implemented quickly. “What I’m suggesting here is it’s a step in the right direction and it’s something that we do quickly,” he said. “I think it’s probably something that China would agree to.”

That last point is significant because the AI safety debate is unfolding alongside an increasingly explicit competition between the United States and China over advanced AI.

Trump has said that slowing the development of U.S. AI systems could allow China to gain an advantage. Amodei has also acknowledged the geopolitical dimension, describing the question of whether to slow development as the “toughest dilemma.”

China’s Foreign Ministry, meanwhile, dismissed the push by AI companies for a slowdown as “fear mongering,” according to a Reuters translation of remarks made Monday.

Musk’s Companies Face Their Own AI Scrutiny

Musk’s call for industry self-regulation also comes as his companies face legal and regulatory disputes over AI.

xAI has challenged AI-related legislation in U.S. states, including California’s AI Training Data Transparency Act, known as AB 2013, and a Minnesota law banning so-called nudify applications.

At the same time, SpaceX’s AI business is facing probes and lawsuits following the use of its Grok image-generation tools to produce and distribute non-consensual sexual imagery, including material depicting child sexual abuse.

Those controversies make Musk’s proposal particularly consequential. Peer review can increase the probability that dangerous behavior is identified before deployment, but its credibility depends on the willingness of companies to expose their own systems to scrutiny and act on findings that could delay a product or impose additional costs.

Musk’s AI empire has also expanded rapidly. SpaceX acquired his AI business xAI in February and completed a $60 billion acquisition of AI code-generation startup Cursor in August. The combined company is working to make SpaceXAI’s Grok models and tools more relevant to developers, putting them in direct competition with products from OpenAI, Anthropic and Google.

That competitive overlap is precisely what makes Musk’s proposed model both potentially useful and difficult to implement. A rival may be well positioned to discover a weakness in another company’s model, but it is also a direct commercial competitor that could benefit from exposing that weakness.

The broader debate is now moving beyond the question of whether AI companies should slow down. The more practical question is whether the industry can create a system in which developers are required to expose powerful models to credible scrutiny before those systems reach millions of users.

Musk’s “test harness” proposal offers one version of that model. Amodei’s embedded evaluators offer another. Both seek to solve the same problem: AI companies have strong incentives to move quickly, while the consequences of a serious failure can extend well beyond any single company.