California governor signs order to explore AI 'kill switch' / Photo: © GETTY IMAGES NORTH AMERICA/AFP/File
California governor signs order to explore AI 'kill switch'
California Governor Gavin Newsom signed an executive order Friday directing state officials to explore the feasibility of imposing a mandatory "kill switch" for the most advanced AI models.
Newsom, widely seen as a possible Democratic presidential contender in 2028, said he was making the move because US President Donald Trump was doing nothing on the issue.
The Republican leader this week dismissed fears over AI as a hoax and is urging US AI companies to keep advancing in order to stay ahead of China in developing the novel technology.
California is home to Silicon Valley and the AI firms at the forefront of the sector.
"The federal government's abject failure to create any form of meaningful AI oversight or accountability should alarm every American, especially when AI CEOs themselves are begging for regulation," Newsom said in a statement.
"We're not waiting to act -- we're going to speed up our work on substantial and responsible AI oversight before it's too late."
The order comes days after dire warnings from industry insiders about the threat the technology poses to humanity.
Jacob Coxon, a former AI researcher who said he worked at Anthropic before resigning, set off the latest round of alarm describing how those building the technology "earnestly believe that it could kill us all by the end of the decade."
Anthropic chief executive Dario Amodei published an essay soon after urging caution.
"We must slow the pace at which we improve the capabilities of AI models," he wrote.
OpenAI chief Sam Altman and tech titan Elon Musk lent their support to the idea, as did Google DeepMind's Demis Hassabis, who himself has called for a US-led global regulatory body for the technology.
Newsom's order does not impose a kill switch. It directs state agencies to report by November 16 on whether one is feasible.
The term "kill switch" refers to the ability to halt an AI system when it is deemed to have malfunctioned or gone off the rails.
The order also asks state officials to weigh requiring AI companies to host independent auditors in their labs.
It also asks officials to consider requiring companies to report loss-of-control incidents, such as those recently disclosed by major developers.
They include a July breach of the AI platform Hugging Face by autonomous agents built on OpenAI models, which escaped a testing environment during an internal evaluation.
The incident, and others like it, have raised fears that the latest AI programs have slipped out of the control of the companies building them.
(A.Nikiforov--DTZ)
Featured