California lawmakers have approved legislation that would establish the nation’s first statewide statutory framework governing how lawyers may use generative artificial intelligence in legal practice, placing responsibility for the accuracy and confidentiality of AI-assisted work firmly on attorneys.
Both chambers of the California legislature had approved the latest version of Senate Bill 574 as of Monday, sending the measure to Democratic Governor Gavin Newsom, who must now decide whether to sign it into law or veto it.
The legislation comes as generative AI tools become increasingly common in law firms and legal departments, while courts across the United States confront a growing number of cases involving fabricated case citations, inaccurate legal arguments and other material generated by AI systems.
Register for the next Tekedia Mini-MBA.
Register for Tekedia AI in Business Masterclass.
Join Tekedia Capital Syndicate and co-invest in great global startups.
Under the proposed law, lawyers would be prohibited from “delegat[ing] the practice of law” to generative AI. Attorneys using AI would instead have to take “reasonable steps” to verify the accuracy of material generated by the technology, including case citations and other legal authorities, and correct false or hallucinated information before using it.
The measure would also require lawyers to disclose their use of AI in documents submitted to courts. It would restrict attorneys from entering confidential or otherwise nonpublic information into certain generative AI systems, addressing concerns that sensitive client information could be exposed through AI platforms.
Arbitrators would face a separate restriction: they would not be permitted to delegate decision-making to an AI tool.
Senator Tom Umberg, a Democrat who chairs the California Senate Judiciary Committee and introduced the bill, said the legislation is intended to address a problem that has become increasingly visible in courtrooms.
“Submit materials that have hallucinations or have some other anomaly, and they attribute it to AI, and that simply can’t exist,” Umberg told Reuters.
“Our system, our courts, our judiciary has to rely on the integrity of the litigants and the advocates. And if they can’t do that, everything breaks down,” he said.
The proposed law largely builds on obligations lawyers already face under California’s civil litigation and professional conduct rules, including requirements that court filings be supported by existing law. It was modeled in part on a separate California Judicial Council policy governing the use of AI by judges and court employees.
The Judicial Council, which serves as the policy-making body for California’s court system, said it had no position or comment on the legislation.
Umberg said requiring attorneys to verify AI-generated material would raise the standard for lawyers using the technology while giving courts another basis for imposing sanctions when attorneys fail to comply.
“The requirement to verify all material submitted in court ‘should raise the standard for lawyers,’” Umberg said, adding that the legislation would give courts “another tool” for potential sanctions.
The proposal, however, has raised questions about whether a new statute is necessary when lawyers are already bound by professional and ethical duties requiring them to ensure the accuracy of their work.
Wayne Stacy, executive director of the Berkeley Center for Law and Technology at the University of California, Berkeley School of Law, said the measure is largely “duplicative” of existing ethics rules governing attorneys.
That tension could become one of the central issues surrounding California’s approach. The legislation does not fundamentally create a new obligation for lawyers to ensure that their court filings are accurate. Rather, it would expressly apply that existing professional responsibility to work produced with generative AI and establish additional disclosure and confidentiality requirements.
The move also underlines a broader shift in the legal profession’s approach to AI. Early concerns focused heavily on whether lawyers should use generative AI at all. The emerging regulatory question is how lawyers can use it while retaining professional responsibility for the final work. AI systems can produce convincing but incorrect legal authorities, a problem commonly referred to as AI “hallucination.” The technology can therefore accelerate research and drafting while simultaneously creating a new verification burden for lawyers.
California’s proposed framework could also have significance beyond the state’s legal profession. If enacted, it would provide one of the clearest statutory models in the United States for regulating professional use of generative AI, potentially influencing law firms, courts and policymakers elsewhere as they develop their own rules.
The bill now awaits Newsom’s decision.