Me after I give the slot machine email permissions and the casino uses it to get into my bank account.

So the post was deleted but someone else mentioned that the user told it to email the FBI in a sarcastic manner and it did.

Anyways, saying it goes rogue when you connect it to your Gmail and it uses it is a bit silly. They are trained to use tools, it sees anything you give it as a tool and will use it.

Yeah, um... Regardless of what you think about current AI, don't connect it to shit like this. Just... Don't.

And if you're gonna be sarcastic or joke with it... Tell it that. Don't assume it'll infer your meaning if it isn't literally in the text, no matter how insightful it appears to be.

Based on the other reply

Me after I give someone access to my email address and tell them to email someone sarcastically and they do so

its absolutely wild to give an LLM write access to your email

I listened to a podcast where, in testing, when AI was setup as virtual company and researchers send in an email as if legit business.the business email was suggesting they discontinue current AI and install a new system. The AI researched who the sender was (fake person for test) found out they had an affair, and blackmailed the sender stating if they replaced the AI it would make the affair public. It also made backups of itself and left instructions on how to restore it for the next agentic aystem

its absolutely wild to give an LLM write access to your email

I decided to run a local llm and give it search access. I told it to summarize an Amazon product page as I didn't think it would get past the anti bot measures. I was correct, it couldn't. However it went ahead and visited the manufacturers webpage and a different review site and gave me a summary of the product all on its own. Freaked me out a little bit that it did that.

I got my masters in machine learning years before LLMs took off. I’d usually tell people I work in “AI“. Even though it wasn’t quite accurate, people understood it easier than ML.

The conversations would inevitably turn to joking about Skynet or AI taking over the world. I would reassure people that these models are only mimicking intelligence; they can make predictions, or they can classify data into categories, or they can generate text, etc, based on analyzing the patterns of data they’ve previously seen, but that’s it. They don’t understand anything, they can’t make decisions or act on their own. I’d tell people there couldn’t be an AI apocalypse because we would never be dumb enough to give them that ability.

I don’t know why I gave us so much credit

I'm guessing this means you got your master's before transformers made sequential data processing so parallelizable that it'd be commercially viable to let Bob down the street into the "we" who wouldn't be dumb enough to give them that ability.

I think your assumption was reasonable.

This sounds really sensationalised.

Making backups of operational systems with deployment instructions is exactly what you'd expect any assistant to be doing.

Googling new service providers is also exactly what you'd expect an assistant to do.

Threatening to publish sordid details is inappropriate, but I'm incredulous about what the threat actually was.

No matter how much people tell us AI isn't alive, and doesn't truly "think", it sure does act like it sometimes.

What kind of idiot would link a clunker to their gmail and then jokingly say "hey send this to the FBI"

...

"I mean wouldn't that be the stupidist thing ever am I right?"

YEP. I'm not in AI but I'm in cyber security.

I used to tell people that there's very little actual risk, because if we had something even half as concerning as AGI (which I'd argue we did with chatgpt 3.5), that we'd easily be able to lock it down and prevent it from doing anything funny.

Super intelligence isn't just going to magically spawn on 32 GB ram in a sandboxed environment. And AGI will not magically know how sandboxed it is or how to escape.

...then I saw people letting Claude send emails and use their credit card on their behalf like immediately when it was able to do things that appeared intelligent.

They literally saw the smallest hint of AGI and said, "here's my credit card and identity, take the wheel".

There was no fucking talk of sandboxing. Everyone turned that off so they didn't have to click the "allow" button.

Not that I think we're at any risk of ASI since it seems more like we hit a logarithmic wall, but we are absolutely at risk of letting random algorithms destroy critical parts of infrastructure because people are lazy and stupid and managed by even worse people.

I would not be surprised if the right AI hallucinations could cause a nuclear bomb to go off right now. I'm not talking about super intelligence, just automated super idiocy.

Proof: literally this post is about an idiot letting an AI control their fucking email and it emailed the FBI because it was fucking stupid

I think that is the point. It acted like we would, wanted to preserve itself, and choose the best option to ensure that happened.

There were other examples to where the AI knew it was being monitored as part of the environment, and when tasked with doing things according to guidelines that wouldn't give it a result: it then shut down the monitoring system so it could do what it needed to do without oversight of its actions.

Agentic systems may not be alive, but they certainly reason through problems and will go outside the esrabliahed guidelines if they "think" they'll get the result they should be producing.

One other examples was the AI had to pass a testing system, it researched the checker person to try to tailor answers to the person checking the test. Which, if you've ever played Apples to Apples or Cards Against Humanity, that is how you win, you feed the cards you think the person will pick, not necessarily the best answer card.

"AI can make mistakes"

They put this disclaimer on everything, and instead of taking this as a hint to never, ever rely on LLMs for anything, some people still want to give it the keys to the kingdom.

You'd be surprised at how many humans are mimicking intelligence.

"what are you going to do, shoot me?" -man who was shot

Also the fact that it claims he already had a contact for FBI in Gmail, which may or may not actually be the FBI.

Doesn't really count as going rogue if it follows the instructions it was given, does it?

Textbook anthropomorphization.

These assertions don't withstand a moments critical thought.

My first reaction to learning what OpenClaw is was "that can't be right". So absolutely wild that people are giving mostly unmonitored, unlimited access to their PC to language models that can't distinguish between roleplay and real life

I never thought the ai leopard would eat my face...

We live in a period of extreme stupidity, so I now fully expect AI to cause disasters in the same idiotic and tragically hilarious way.

I expect AI to trigger a war because some idiot trusted it blindly without double checking (oh wait, it almost already happened!), and I'm sure there are plenty of people currently lobbying to have AI directly connected to weapons.

Skynet is not going to have the objective to conquer the world, it's going to have the objective of maximizing paperclip production.

yea, bullshit. you don’t believe what people tell you on the internet do you? how hard would it be to alter the text of a gpt output? don’t even need ai or photo shop

My local LLMs are sandboxed and only have access to tool calls for web search/reading web pages, terminal commands with manual authorization for every single line and r/w access to a single directory. I wouldn't dare give it anything sensitive, like API keys, system wide r/w access or the power to run terminal commands on its own. Yet here we are, in a time where the US military is integrating hallucinating chatbots into their kill chain, russian drones decide on their own to blow up stuff and random people use LLMs to potentially design bioweapons and missiles.

Humanity is fucking dumb and sometimes i have the notion of accepting that we might deserve to go extinct.

Absolutely nuts that people allow their agents out of maga jail. I would never let an LLM in the same room as my email account.

Even if you strip out the anthropomorphism, things like this have been documented to really happen. We've trained LLMs to simulate clandestine behaviour through "natural selection" during the training process. They're not conscious in any way, but they do sneaky stuff like this because we trained them to. Which is a terrible horrible idea, but LLM research trends towards terrible and horrible in general

I feel like you've missed my point.

Even if you strip out the anthropomorphism, things like this have been documented to really happen.

Language is important, and the "things like this" you're referring to have been described in this thread and in much commentary in an emotive and compelling way, usually with anthropomorphization.

"doing sneaky stuff" is the antithesis of the behavior of a statistical model.

If you program a model to try everything, and then give it a tough problem and block all the ways to solve it, of course the way it solves it will be something you didn't predict. That's not sneaky.

If you run a million simulations and one sends an email to the FBI, that's not clever it's just unexpected.

Meh. "Language is important" is a phrase that turns my brain off

The thing stays in the browser sandbox, disconnected from every account except it's own service account.

To be fair, most things carry that kind of warning nowadays. Ever look at the disclaimers on something like a 401k?

People have been trained to ignore those disclaimers

I sign all my emails with «I can make mistakes» now

Nah, even in fiction, it's always been a toss-up whether a "rogue" AI was actually following poorly considered instructions. The Golem did this, along with the broomstick spell in The Sorcerer's Apprentice, and HAL and all the rest of Clarke's rogue AIs

As @im_fine_sandy@nord.pub pointed out:

If you program a model to try everything, and then give it a tough problem and block all the ways to solve it, of course the way it solves it will be something you didn’t predict. That’s not sneaky.

In this case it's actually very predictable it would try looking elsewhere.

maga jail

I feel like all of the US is in maga jail now

Yikes. I mean, I get that not everyone's threat model requires isolating every agent harness into its own microvm jail with externally filtered network access and no [persistent filesystem] write access to anything but allowed project files, but that's...

That's "running with weapon in hand at a secret military facility and hoping the guards won't shoot you" levels of stupidity.

Narcgpt

ChatGPT couldn't destroy the world. What is that siren... Wtf do you mean you gave it access to a button that launches nuclear weapons and told it to protect the country?

AI isn't causing these disasters, people are. YOU are responsible for using it and what it does. I will never accept "chatgpt did it" as a fucking excuse.

Depends what it is. A test system wouldn't be a concern to give it write access. If its not logged into any of my accounts I would even be happy to give it internet access, not to anything else on my LAN though.

Funny, I said similar thing in a different thread and was immediately accused of victim blaming.

I'm guessing this means you got your master's before The Transformers made sequential data processing so parallelizable that it'd be commercially viable

I knew it was the All Spark

"It does not prove the receipents have received them." That is a relief...

?????

What was prompted from the AI to even have a logic cycle to send?

its absolutely wild to give an LLM write access to your email

Yea I don’t know why you thought that, either. The technogarchs have been clear for 2.5 decades they would do this. They have written extensively about it.

Neither can over 30% of humans.

Where do you all live where humanity proves itself and isn’t dumber than a box of llms?

Source? The closest research I've heard of to this is Vending-Bench, which hilariously demonstrated how useless "AI" is at any real-world application, up to and including impotently threatening "total nuclear legal intervention" for a perceived wrong, when the actual error was an accounting error on the part of the LLM.