A 27-year-old artificial intelligence researcher who just left AI firm Anthropic has spoken out, saying that many engineers and scientists developing the technology are extremely fearful of its rapid development and the existential threat it could pose to humanity.
Jacob Coxon went viral with his resignation article on the perils of unregulated AI, making headlines. He told the BBC in an interview that without a conscious slowing down of development there was a major danger of cataclysmic events soon enough.The Drive to Slow Down
Coxon’s old boss, Anthropic CEO Dario Amodei, recently called for a slowdown in AI research to give society and authorities time to catch up with the threats. That view is shared by competing tech luminaries such Sam Altman of OpenAI and Elon Musk of xAI, who are calling for industry-wide regulation and independent monitoring.
Coxon supported the idea of slowing down but insisted that it must be done in a coordinated international manner and that China must be included so as to prevent an uncontrolled international race. “They are deadly serious when they ask for regulation because they are trapped in a race,” he said of the people who work at the corporations.
An 'Apocalyptic' Time Frame
Internally, the mental toll is enormous. Coxon said AI professionals often think about the consequences of their work, with some even considering buying land or preparing for a major societal breakdown. He said worst-case scenarios like rogue AI bots taking over the internet might become realistic in six months to a year.
These worries are shared by other high level people. AI scientist Evan Hubinger put the odds of AI killing off humanity in the decade at more than 10%, a chance that was “not unreasonable” according to computer science pioneer Geoffrey Hinton (often dubbed the “Godfather of AI”). Former UK Prime Minister Rishi Sunak, an adviser to Anthropic, has also raised similar alarm bells.
Anthropic Defense
In reaction to the departure, an Anthropic spokesman pointed to the company’s dedication to safety, saying they were the first to create frameworks to mitigate development risks, rigorously stress-test models, and push hard for a legitimate, verifiable approach to pacing the rollout of powerful systems.
Silicon Valley Skepticism
But not everyone in the computer community is convinced by the doomsday warnings. Critics say these doomsday stories are either overhyped or deliberately concocted to create buzz and keep investors’ prices up.
“The way we ask AI researchers about the risks of extinction is like asking an air-conditioning technician about climate change,” said Clement Delangue, CEO of Hugging Face. Grindr CEO George Arison has argued that these claims assist to justify the sky-high valuations of companies, suggesting that they are poised to take over every industry. Reports have also said that Nvidia CEO Jensen Huang has denied the charges, while political figures including US President Donald Trump have highlighted that the key goal is to win the global AI race against rivals like China.
An 'Apocalyptic' Time Frame
Internally, the mental toll is enormous. Coxon said AI professionals often think about the consequences of their work, with some even considering buying land or preparing for a major societal breakdown. He said worst-case scenarios like rogue AI bots taking over the internet might become realistic in six months to a year.
These worries are shared by other high level people. AI scientist Evan Hubinger put the odds of AI killing off humanity in the decade at more than 10%, a chance that was “not unreasonable” according to computer science pioneer Geoffrey Hinton (often dubbed the “Godfather of AI”). Former UK Prime Minister Rishi Sunak, an adviser to Anthropic, has also raised similar alarm bells.
Anthropic Defense
In reaction to the departure, an Anthropic spokesman pointed to the company’s dedication to safety, saying they were the first to create frameworks to mitigate development risks, rigorously stress-test models, and push hard for a legitimate, verifiable approach to pacing the rollout of powerful systems.
Silicon Valley Skepticism
But not everyone in the computer community is convinced by the doomsday warnings. Critics say these doomsday stories are either overhyped or deliberately concocted to create buzz and keep investors’ prices up.
“The way we ask AI researchers about the risks of extinction is like asking an air-conditioning technician about climate change,” said Clement Delangue, CEO of Hugging Face. Grindr CEO George Arison has argued that these claims assist to justify the sky-high valuations of companies, suggesting that they are poised to take over every industry. Reports have also said that Nvidia CEO Jensen Huang has denied the charges, while political figures including US President Donald Trump have highlighted that the key goal is to win the global AI race against rivals like China.
I, too, am full of apprehension that the situation will spiral out of control and that what we call the future today could turn out to be our worst nightmare—or perhaps even the end. This is highly probable.
ReplyDeleteIn my opinion, the development of artificial intelligence is moving too fast. We humans are not prepared for such a "leap." Melody, while discussing this topic, I’d like to raise another issue: the development of artificial intelligence is taking jobs away from people. Across various professions and skill sets, human workers will no longer be needed because robots and computers will replace them. This is a massive problem—the impoverishment of society (often drastically so). People will be out of work and lose their ability to earn a living. I have to ask: how will these people survive? People need to eat and support their families. They have to pay taxes and bills for gas, electricity, and water... What happens if they or their loved ones fall ill? These are just some of the problems associated with the development of artificial intelligence.
ReplyDeleteSi que escuche esta noticia y creo que aunque no debemos poner puertas al campo si que sería bueno una regulación del uso de la IA. Ya como se ve en ciertas aplicaciones teniendo un buen uso es beneficioso en otros puede ser muy perjudicial.
ReplyDeleteSaludos.