California governor vetoes major AI safety bill

California Governor Gavin Newsom vetoed the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act (SB 1047) today. In his veto message, Governor Newsom cited multiple factors in his decision, including the burden the bill would have placed on AI companies, California’s lead in the space, and a critique that the bill may be too broad.

“While well-intentioned, SB 1047 does not take into account whether an AI system is deployed in high-risk environments, involves critical decision-making or the use of sensitive data. Instead, the bill applies stringent standards to even the most basic functions — so long as a large system deploys it. I do not believe this is the best approach to protecting the public from real threats posed by the technology.”

Newsom writes that the bill could “give the public a false sense of security about controlling this fast-moving technology.”

“Smaller, specialized models may emerge as equally or even more dangerous than the models targeted by SB 1047 – at the potential expense of curtailing the very innovation that fuels advancement in favor of the public good.”

Governor Newsom also writes that he agrees that there should be safety protocols and guardrails in place, as well as “clear and enforceable” consequences for bad actors. However, he states that he doesn’t believe the state should “settle for a solution that is not informed by an empirical trajectory analysis of Al systems and capabilities.”

Here is the full veto message:

It would have applied to covered AI companies doing business in California with a model that costs over $100 million to train or over $10 million to fine-tune, adding requirements that developers implement safeguards like a “kill switch” and lay out protocols for testing to reduce the chance of disastrous events like a cyberattack or a pandemic. The text also establishes protections for whistleblowers to report violations and enables the AG to sue for damages caused by safety incidents.

Changes since its introduction included removing proposals for a new regulatory agency and giving the state attorney general power to sue developers for potential incidents before they occur. Most companies covered by the law pushed back against the legislation, though some muted their criticism after those amendments.

The Chamber of Progress, a coalition that represents Amazon, Meta, and Google, similarly warned the law would “hamstring innovation.”

The bill’s opponents have included former House Speaker Nancy Pelosi, San Francisco Mayor London Breed, and eight congressional Democrats from California. On the other side, vocal supporters have included Elon Musk, prominent Hollywood names like Mark Hamill, Alyssa Milano, Shonda Rhimes, and J.J. Abrams, and unions including SAG-AFTRA and SEIU.

The federal government is also looking into ways it could regulate AI. In May, the Senate proposed a $32 billion roadmap that goes over several areas lawmakers should look into, including the impact of AI on elections, national security, copyrighted content, and more.