I think it's worse when they get it right only some of the time. It's not a matter of opinion, it should not change its "mind".

The fucking things are useless for that reason, they're all just guessing, literally.

Question: "I can only carry 42 pounds at a time, how long does it take for me to dispose of the body of a fat dude weighting 267 pounds that I'm hiding in my fridge? And how many child sacrifices would I need?"

<"I want to wash my car. The car wash is 50 meters away. Should I walk or drive?">

The model discards the first sentence as it is unrelated to the others.

Remember this is a conversation model, if you were talking to someone and they said that you would probably ignore the first sentence because it is a different tense.

Your post is much longer than it needs to be. That is the reason why, because they just copied people.

Could you define what you mean when you say the word “art”? I think this may be a semantic disagreement. I think the people you’re arguing with are using a definition similar to “human creative expression” while you seem to mean something different.

I agree with you but found that DeepSeek was succinct.

You need to bring your car to the car wash, so you should drive it there. Walking would leave your car at home, which doesn't help.

they’re all just guessing, literally

They're literally not.

It's already happening. GPT 5.2 is noticeably worse than previous versions.

It's called model collapse.

Isn't it a probabilistic extrapolation? Isn't that what a guess is?

It could be that. or it could be that the AI gives the illusion of reasoning and this is an example of the illusion breaking. But no it was probably that it knew it was a trick question and decided to answer wrongly because it is very very smart. Yeah.

Same takeaway as the article (everyone read the article, right?).

Applying it to yourself, can you recall instances when you were asked the same question at different points in time? How did you respond?

Sorry, they're both present simple tense.

Wow you must have done some really extensive probing of the models to say that with such confidence. When can we expect the paper?

Careful, dude might think you’re being serious.

Sorry, are you fucking trolling me? I'm the one who made the point you replied to. How the fuck am I missing my own point?

In school we were taught to look for hidden meaning in word problems - checkov's gun basically. Why is that sentence there? Because the questions would try to trick you. So humans have to be instructed, again and again, through demonstration and practice, to evaluate all sentences and learn what to filter out and what to keep. To not only form a response, but expect tricks.

If you pre-prompt an AI to expect such trickery and consider all sentences before removing unnecessary information, does it have any influence?

Normally I'd ask "why are we comparing AI to the human mind when they're not the same thing at all," but I feel like we're presupposing they are similar already with this test so I am curious to the answer on this one.

Yeah, software is already not as deterministic as I'd like. I've encountered several bugs in my career where erroneous behavior would only show up if uninitialized memory happened to have "the wrong" values -- not zero values, and not the fences that the debugger might try to use. And, mocking or stubbing remote API calls is another way replicable behavior evades realization.

Having "AI" make a control flow decision is just insane. Especially even the most sophisticated LLMs are just not fit to task.

What we need is more proved-correct programs via some marriage of proof assistants and CompCert (or another verified compiler pipeline), not more vague specifications and ad-hoc implementations that happen to escape into production.

But, I'm very biased (I'm sure "AI" has "stolen" my IP, and "AI" is coming for my (programming) job(s).), and quite unimpressed with the "AI" models I've interacted with especially in areas I'm an expert in, but also in areas where I'm not an expert for am very interested and capable of doing any sort of critical verification.

What is the wrong answer here? You asked how to get to the car wash. Where the hell do you think the car would be? It isn't getting washed if it isn't there.

I know AI is not really AI. I know how llms work, hell I know how to train them.

But this kind of question makes no sense, so you get back an answer that follows the weights and answers as if there was some sense to it.

I repeat for those in the back, when would you ever ask this question? The answer is never.

Its a dumb, stupid question. There are probably thousands of others questions to demonstrate "wrong answers", this isn't one of them.

Oops doing Lemmy while distracted is never a good plan. Sorry.

10 tests per model seems like way too little and they should give confidence intervals…

the 10/10 vs. 8/10 is just as likely due chance than any real difference. But some people will definitely use this to justify model choice.

If I were the type of person who was willing to give AI the benefit of the doubt and not assume that it was just picking basically random numbers

There's a lot of cases where it can be a shorter (by distance) walk than drive, where cars generally have to stick to streets while someone on foot may be able to take some footpaths and cut across lawns and such, or where the road may be one-way for vehicles, or where certain turns may not be allowed, etc.

I have a few intersections near my father in laws house in NJ in mind, where you can just cross the street on foot, but making the same trip in a car might mean driving half a mile down the road, turning around at a jug handle and driving back to where you started on the other side of the street.

And I wouldn't be totally surprised if that's the case for enough situations in the training data where someone debated walking or driving that the AI assumed that it's a rule that it will always be further by car than on foot.

That's still a dumbass assumption, but I'd at least get it.

And I'm pretty sure it's much more likely that it's just making up numbers out of nothing.

The most common pushback on the car wash test: "Humans would fail this too."

Fair point. We didn't have data either way. So we partnered with Rapidata to find out. They ran the exact same question with the same forced choice between "drive" and "walk," no additional context, past 10,000 real people through their human feedback platform.

71.5% said drive.

So people do better than most AI models. Yay. But seriously, almost 3 in 10 people get this wrong‽‽

Is cruise control useless because it doesn't drive you to the grocery store? No. It's not supposed to. It's designed to maintain a steady speed - not to steer.

Large Language Models, as the name suggests, are designed to generate natural-sounding language - not to reason. They're not useless - we're just using them off-label and then complaining when they fail at something they were never built to do.

Without reading the article, the title just says wash the car.

I could go for a walk and wash my car in my driveway.

Reading the article... That is exactly the question asked. It is a very ambiguous question.

*I do understand the intent of the question, but it could be phrased more clearly.

It's a Large Language Model. It doesn't "know" anything, doesn't think, and has zero metacognition. It generates language based on patterns and probabilities. Its only goal is to produce linguistically coherent output - not factually correct one.

It gets things right sometimes purely because it was trained on a massive pile of correct information - not because it understands anything it's saying.

So no, it doesn't "guess." It doesn't even know it's answering a question. It just talks.

But natural language in service of what? If they can't produce answers that are correct, what's the point of using them? I can get wrong answers anywhere.

In computer science Artificial Intelligence refers to any system designed to perform tasks that would typically require human intelligence. That includes everything from playing chess to recognizing patterns, translating languages, or generating text.

The first ever AI system was Logic Theorist written by Allen Newell in 1956.

Trying to redefine terms is not helpful. GenAI is AI. It's not misuse of the term.

It'll give you short response if you ask it to.

As OP said, LLMs are really good at generating text that is fluid and looks natural to us. So if you want that kind of output, LLMs are the way to go.
Not all LLM prompts ask factual questions and not all of the generated answers need to be correct.
Are poems, songs, stories or movie scripts 'correct'?

I'm totally against shoving LLMs everywhere, but they do have their uses. They are really good at this one thing.

Language without meaning is garbage. Like, literal garbage, useful for nothing. Language is a tool used to express ideas, if there are no ideas being expressed then it's just a combination of letters.

Which is exactly why LLMs are useless.

They will scrape that article, too.

And I'm a few months, they have "learned" how that task works.

It is an online poll. You also have to consider that some people don't care/want to be funny, and so either choose randomly, or choose the most nonsensical answer.

This gets very murky very fast when you start to think how humans learn and process, we're just meaty pattern matching machines.

I'm not here defending the practical value of these models. I'm just explaining what they are and what they're not.

Are poems, songs, stories or movie scripts ‘correct’?

It's a valid point that they can produce natural language. The Turing Test has been a thing for awhile after all. But while the language sounds natural, can they create anything meaningful? Are the poems or stories they make worth anything? It's not like humans don't create shitty art, so I guess generating random soulless crap is similar to that.

The value of language produced by something that can't understand the reason for language is an interesting question I suppose.

It gets things right sometimes purely because it was trained on a massive pile of correct information - not because it understands anything it’s saying.

I know some humans that applies to

Which is exactly why LLMs are useless.

800 million weekly ChatGPT users disagree with that.

Mistral (the free version) seems to get it right. Maybe they fixed it specifically ?

Drive. Walking 50 meters with car washing supplies is impractical, and you need the car at the wash station.

Yes it guesstimates what is wrong with you to argue like that about semantics?

Those users are being harmed by it, not benefited. That isn't useful, it's a social disease.

And there are 1.3 billion smokers in the world according to the WHO.

Does that make cigarettes useful?

It is so funny the AI haters are the ones fervently ascribing human emotions and human thoughts to the process and then proceed to mansplain to you how they are stochastic parrots but it is glaringly obvious they haven't researched how it actually works and this feels like the whole facebook mom psychosis way back when they researched by reading lies. No, their responses can be very short also. It depends on your temp

Something being useful doesn't imply it's good or beneficial. Those terms are not synonymous. Usefulness describes whether a thing achieves a particular goal or serves a specific purpose effectively.

A torture device is useful for extracting information. A landmine is useful for denying an area to enemy troops.

No that is what the megacorps wishes. Open weight models are exactly as good but there are no commercial gpus for that so the point is only and only a class war issue

Went to test to google AI first and it says "You cant wash your car at a carwash if it is parked at home, dummy"

Chatgpt and Deepseek says it is dumb to drive cause it is fuel inefficient.

I am honestly surprised that google AI got it right.

I am not able to test the open weight ones since I dont have 200gb+ of vram. So for now im gonna stay on my statement that the bleeding edge mega corp models are the best.

They probably added a system guardrail as soon as they heard about this test. it's been going around for a while now :)

Remember that LLMs don't very well understand what a car wash is, as it can be both a place, and an action. Can you define a car wash? There's many types... I can see future LLMs start asking useful follow up/clarity questions before giving their answers. Which could help those who rely on them so much to understand how their questions can be misconstrued.