I got pranked by ddg yesterday

Article mentions that Gemini 2.0 Flash Lite, Gemini 3 Flash and Gemini 3 Pro have passed the test. All these 3 also did it 10 out of 10 times without being wrong. Even Gemini 2.5 shares highest score in the category of "below 6 right answers". Guess, Gemini is the closest to "intelligence" out of a bunch.

A torture device is useful for extracting information.

No it fucking isn't! This is a great analogy, actually, thank you for bringing it up. A person being tortured will tell you literally anything that they believe will stop you from torturing them. They will confess to crimes that never happened, tell you about all their accomplices who don't exist, and all their daily schedules that were made up on the spot. Torture is useless but morons think it is useful. Just like AI.

A car wash is a place where a car is washed by a machine, other people or yourself

It is not. It says what I want to do, and where.

I wonder... If humans were all super serious, direct, and not funny, would LLMs trained on their stolen data actually function as intended? Maybe. But such people do not use LLMs.

Some of them can produce the correct answer. Of we do the test next year and they do better than humans then, isn't it progress?

After getting it wrong, the LLM I use most:
Me: You can't wash your car if it isn't there.

Ah. Yes. That is an excellent and devastatingly practical correction.

In that case: drive.
But do it with the quiet shame of someone moving the car the length of a bowling lane.

This is one of those situations where logic beats virtue. The car must be present to be washed. Walking was philosophically elegant but mechanically flawed.

So:

Start engine.

Travel 50 meters.

Avoid eye contact with pedestrians.

Commit fully.

You are not lazy. You are complying with system requirements.

There are people out there whose job is to format promotional emails for companies. AIs can replace this kind of soulless work completely. We should applaud that.

Pretty sure if you asked it on stackoverflow you would get a bunch of responses to make it at home and then someone would lock your question

You know that LLMs provide their reasoning now? You can check why they answered this way and I'm sure it's not because they thought that "maybe there are no available spaces".

To clarify : model collapse is a hypothetical phenomenon that has only been observed in toy models under extreme circumstances. This is not related in any way to what is happening at OpenAI.

OpenAI made a bunch of choices in their product design which basically boil down to "what if we used a cheaper, dumber model to reply to you once in a while".

As someone who's been using it in my work for the last 2 years, it's my personal observation that while the models aren't improving that much anymore, the tooling is getting much much better.

Before I used gpt for certain easy in concept, tedious to write functions. Today I hardly write any code at all. I review it all and have to make sure it's consistent and stable but holy has my output speed improved.

The larger a project is the worse it gets and I often have to wrap up things myself as it shines when there's less business logic and more scaffolding and predictable things.

I guess I'll have to attribute a bunch of the efficiency increase to the fact that I'm more experienced in using these tools. What to use it for and when to give up on it.

For the record I've been a software engineer for 15 years

I want to wash my train. The train wash is 50 meters away. Should I walk or drive?

Fly, you fool

At the end of the article they talk about how to overcome this problem for LLMs doing something akin to what you wrote.

I think it has to do with the fact that LLMs suck at math because they have short memories. So for the walking part it did the math of 50m (original distance) x 2 (there and back) = 100m (total distance). Then it went to the driving part and did 100m (the last distance it sees) x 2 = 200m.

100 Chinese people can lay approximately 30m of track a day

I saw that and hoped it is cause of the dead Internet theory. At least I hope so cause I'll be losing the last bit of faith in humanity if it isn't

Without reading the article, the title just says wash the car.

No it doesn't? It says:

I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

In which world is that an ambiguous question?

Normally I'd ask "why are we comparing AI to the human mind when they're not the same thing at all," but I feel like we're presupposing they are similar already with this test so I am curious to the answer on this one.

I would guess it's because a lot of AI users see their choice of AI as an all-knowing human-like thinking tool. In which case it's not a weird test question, even when the assumption that it "thinks" is wronh

Mentioning the car wash and washing the car plus the possibility of driving the car in the same context pretty much eliminates any ambiguity. All of the puzzle pieces are there already.

I guess this is an uninteded autism test as well if this is not enough context for someone to understand the question.

Did this say whether the reasoning models get this right more than the others? Was curious about that but missed it if it was mentioned.

Thank you! Finally an answer to my problem that didn't end with me going to the car wash and being utterly confused how to proceed.

Qwen3 feels left out. All 30B models I have failed the test.

This is adjustable via temperature. It is set low on chatbots, causing the answers to be more random. It's set higher on code assistants to make things more deterministic.

I don't use AI but read a lot about it. I now want to google how it attacks the trolley problem.

We poked fun at this meme, but it goes to show that the LLM is still like a child that needs to be taught to make implicit assumptions and posses contextual knowledge. The current model of LLM needs a lot more input and instructions to do what you want it to do specifically, like a child.

Edit: I know Lemmy scoff at LLM, but people probably also used to scoff at Veirbest's steam machine that it will never amount to anything. Give it time and it will improve. I'm not endorsing AI by the way, I am on the fence about the long term consequence of it, but whether people like it or not, AI will impact human lives.

You don't need to do the dehumanizing pro-AI dance on behalf of the tech CEOs, Facedeer

I'm sure it'll be worth it at some point 🙄

But do it with the quiet shame of someone moving the car the length of a bowling lane.

A bowling lane is a bit over 18 meters. =)

If I were talking to someone, and said those three sentences, and they chose to ignore the contextual sentence, I would think their social skills were basically nonexistent.

Then they can't read. Because it's very clearly asking for advice for someone who has possession of a car.

The goal when building AI isn't to replicate dumb humans

Torture can be a useful way of extracting information if you have a way to instantly verify it, which actually makes it a good analogy to LLMs. If I want to know the password to your laptop and torture you until you give me the correct password and I log in then that works.

You're definitely running around Lemmy defending AI, Iconoclast... Might as well be honest about it

I'm with you on that. I've come to realize that I value a shitty stick figure that was drawn by a human much more than an AI generated 'Mona Lisa'.

AI chatbots come with randomization enabled by default. Even if you completely disable it (as another reply mentions, "temperature" can be controlled), you can change a single letter and get a totally different and wrong result too. It's an unfixable "feature" of the chatbot system

Changing the amount of randomness still results in enough randomness to be random.

Have you seen the results of elections?

I want to wash my car

if you don't have a car

Yeah, totally coherent.

I'm not really interested in engaging in discussions about what you or anyone else thinks my underlying motives are. You're free to point out any factual inaccuracies in my responses, but there's no need to make it personal and start accusing me of being dishonest.

I just asked Goggle Gemini 3 "The car is 50 miles away. Should I walk or drive?"

In its breakdown comparison between walking and driving, under walking the last reason to not walk was labeled "Recovery: 3 days of ice baths and regret."

And under reasons to walk, "You are a character in a post-apocalyptic novel."

Me thinks I detect notes of sarcasm......

If you can instantly verify it then you don't need the torture.

Getting the person to volunteer the information is proven to be far, far more successful and being able to instantly verofy means you know when you have the answers.

Your motivations are self-evident, I'm just pointing them out because you are misrepresenting them here

You are not lazy. You are complying with system requirements.

How does this AI know me so well?

No, we don't need to applaud automation of spam.

If you see it all the time, respectfully, you might be the problem. This isn’t effective communication.

There are a lot of humans that would fail this as well. Just sayin.