California Governor Gavin Newsom has vetoed a landmark bill that aimed to establish some of the first regulations on large-scale artificial intelligence (AI) models in the US The decision, made on Sunday, marks a significant setback for those pushing to rein in the rapidly growing tech sector, which has so far evolved with minimal oversight.
The bill would have introduced safety measures for large AI models, setting the stage for similar regulations across the country. Supporters argued that these regulations were necessary to address potential risks associated with advanced AI technology. However, Newsom expressed concerns that the proposed law could stifle the homegrown industry and hinder innovation.
Earlier in September, Newsom had suggested during Salesforce’s Dreamforce conference that California should take the lead on AI regulation due to federal inaction. However, he also noted that the proposed bill could have a “chilling effect” on the industry. The bill had faced strong opposition from tech giants, startups, and several Democratic House members, who argued it would impose overly rigid requirements on AI development.
Concerned over broad applications
In his veto statement, Newsom said that while the bill was well-intentioned, it applied stringent standards to AI systems across the board, without considering whether those systems were being used in high-risk environments or for sensitive tasks. He argued that the bill placed unnecessary burdens even on basic AI functions, which he did not believe was the best approach to ensuring public safety.
Instead, Newsom announced a partnership with several AI industry experts, including AI pioneer Fei-Fei Li, to create more nuanced safety guidelines around AI models. Li, who opposed the bill, will help develop these guardrails.
The bill, authored by Democratic state Senator Scott Weiner, aimed to reduce the potential risks AI poses by requiring companies to test their models and disclose safety protocols. This would have included measures to prevent AI systems from being manipulated for dangerous purposes, such as disabling critical infrastructure or aiding in the creation of chemical weapons. The bill also offered whistleblower protections to employees.
Debate sparks further discussions
Senator Weiner called the veto a blow to public safety, stating that the lack of oversight leaves corporations unchecked as they develop powerful AI systems that could impact the future of society. He acknowledged that while the veto is disappointing, the debate has advanced the conversation around AI safety. Weiner pledged to continue advocating for stronger AI regulations.
The bill was one of several legislative efforts this year aimed at regulating AI in California, including measures to combat deepfakes and protect workers. Lawmakers have expressed concerns about the lessons learned from failing to regulate social media early on and are keen not to repeat the same mistakes with AI.
Supporters of the bill, including Elon Musk and AI research firm Anthropic, argued that transparency and accountability are crucial as AI technology advances. They pointed out that even AI developers and experts don’t fully understand how some of the most powerful AI models behave. The bill would have targeted systems that require significant computing power and financial resources to build, which, while not common now, are expected to increase in the near future.
Tech industry pushes back
The tech industry, however, pushed back hard against the bill, with critics like former House Speaker Nancy Pelosi arguing that it would “kill California tech” by discouraging investment in AI development. Concerns were raised that the bill could stifle innovation and make developers less likely to share open-source AI software.
The veto marks a win for the tech industry, which had spent the past year lobbying alongside the California Chamber of Commerce to influence lawmakers and the governor. Two other AI-related bills also failed to pass before the legislative deadline, including one that would have required AI-generated content to be labelled and another aimed at banning discrimination from AI tools used in employment decisions.
A balancing act
Governor Newsom has made it clear that he wants California to remain a global leader in AI development. He noted that 32 of the world’s top 50 AI companies are based in the state and has promoted California as an early adopter of generative AI tools to tackle issues like highway congestion, tax guidance, and homelessness programs.
In the weeks leading up to the veto, Newsom signed other significant AI-related legislation, including some of the toughest laws in the country to combat election deepfakes and protect Hollywood workers from unauthorised AI use.
While the AI safety bill has been blocked, experts believe it could inspire similar measures in other states. Tatiana Rice, deputy director of the Future of Privacy Forum, said that the bill’s ideas are likely to resurface in future legislative sessions, as concerns over AI risks continue to grow.
For now, California remains at the forefront of the AI debate, balancing the need for innovation with the call for responsible development.