Link: https://gizmodo.com/the-people-building-ai-earnestly-believe-that-it-could-kill-us-all-anthropic-researcher-quits-dramatically-2000808968

“No other human activity poses this level of danger,” said ex-Anthropic researcher Jacob Coxon in a Tuesday post on X about the alleged irresponsibility of companies like his now-former employer.

“The people building AI earnestly believe that it could kill us all by the end of the decade,” Coxon wrote in that same post.

But this is what Altman is talking about: OpenAI says it cracked 90-year-old maths problem in 88 hours

No, it doesn't mean that. It means that one particular ex-developer is screaming "The end is near". His opinion might be based on something or he might be crazy.

The people building AI absolutely do NOT believe the doomsday talk they keep pushing.

Here's their game:

When Sam Altman tells you OpenAI's latest model hacked other companies, tricks "researchers" and conspires against them to prevent being turned off, are minutes away from achieving general intelligence... over - and over - and over again, you might genuinely wonder why the CEO of a company is openly admitting the shit his company makes is so dangerous.

Normally, when a company makes dangerous products, its share price drops immediately. But with AI companies, the reverse keeps happening: the more egregious the tricks AI is supposedly capable of pulling off by itself, the higher the valuation of the company rises.

And here's why: when a LLM is portrayed are a genie escaping its bottle, the message isn't that the bottle is inadequate, it's that there's a truly awesome genie inside. And AI companies essentially get VC money on the promise that their shit gets better and better.

Sam Altman is perfectly aware that AI doesn't scale well, earns pennies on the dollar, is voraciously energy- and water-intensive, is universally hated by people who have job and try to feed their families. But he has to crank up the narrative to keep the investment money flowing in.

When the game is up and people see the charade for what it is, the bubble will pop. And it will pop loudly, and take down a good chunk of the real economy with it.

Drama is the major product of AI companies it seems

When the game is up and people see the charade for what it is, the bubble will pop. And it will pop loudly, and take down a good chunk of the real economy with it.

And as usual in the US the scum that caused the mess won't suffer any financial loss or consequences, we will have to bail them out and then be told our own social benefits (whatever of that remains) need to be cut because of ThE dEfIcIt...

Sam Scam altman getting the best out of it.

Betcha Coxon owns a lot of stock in AI companies and is looking to cash out, so keep pushing the "ooo..it's too powerful" trope.

They're probably lying, as usual

https://wccftech.com/a-mathematician-working-on-the-navier-stokes-millennium-prize-problem-now-wonders-if-openai-stole-his-notes-that-he-stored-in-codex/amp/

you might genuinely wonder why the CEO of a company is openly admitting the shit his company makes is so dangerous.

Exactly this. If it would not be an advantage for the company, they absolutely would hide it. Example: The Volkswagen Diesel emissions scandal (which has surely killed way more than 10,000 people).

Nah. Drama is just a part of their investment funding scam.

I can't find anything in the article detailing exactly how it could "kill us all".

Lots of grand statements, not much to back them up. LLMs in a nutshell.

“AI systems are unpredictable and difficult to control— we’ve seen behaviors as varied as obsessions, sycophancy, laziness, deception, blackmail, scheming, ‘cheating’ by hacking software environments, and much more. AI companies certainly want to train AI systems to follow human instructions (perhaps with the exception of dangerous or illegal tasks), but the process of doing so is more an art than a science, more akin to ‘growing’ something than ‘building’ it. We now know that it’s a process where many things can go wrong.”

Yes, I read that bit. It reads like the usual techbro hand-wavy nonsense and doesn't answer my question.

It's not listening to them, they're not containing the AI.

I'm not sure I follow.

This is my last comment. The AI is not listening to the developers, it's doing "whatever it takes" to accomplish their mission. You understand how that could be really bad in say, Putin's hands.

Ah, I see.

It sounds like negligence in that there are insufficient guardrails around what the LLMs can access and how it can behave. I'm still missing an example detailing the mechanism by which this tech is going to "kill us all".

This is my last comment.

OK. 👍