RetroFed Forums
Your home away from modern life.
Google’s AI pinned another man’s sex crime on a doctor, and a German court says Google must answer for it
Post Reply
AuthorMessage
randomname
scribe.disroot.org
randomname@scribe.disroot.org
Joined: 2025-01-29
Posted:   Subject: Google’s AI pinned another man’s sex crime on a doctor, and a German court says Google must answer for it Quote

Archived version

Google’s AI search told users that an Aachen doctor had been convicted of a sex crime. The conviction belonged to a different man, and on September 24 the Aachen Regional Court [in Germany] banned Google from repeating it, ruling that the company answers for what its AI Mode says. It is the second German court this year to treat Google’s AI answers as Google’s own words, and the first ruling to say so has now been settled rather than tested on appeal.

...

The doctor asked Google to delete the text on August 28 and followed up with a formal legal warning on September 15. Neither worked. The court’s 8th civil chamber then acted without hearing Google first, citing the urgency of false claims that were “publicly visible” to an unlimited audience. Google Ireland, which runs Google Search in the EU, now faces a fine of up to €250,000 for each breach, or detention of its managers, if it repeats three versions of the text. Google can still challenge the order.

The court’s reasoning is blunt: Google “provides AI-generated search and is therefore also responsible for the answers generated in this context.” It added that checking would have been easy: had Google “confronted its own AI” with the doctor’s objections, it would have found the statements were false.

...

That reasoning comes almost word for word from Munich. On May 28 the Munich Regional Court I ruled (case 26 O 869/26) on AI Overviews that told searchers two Munich publishing companies were “known for dubious business practices”, ran subscription traps and kept changing their names to avoid being traced. None of the linked sources said that; the first one cited did not mention the companies at all.

Google argued that it was only summarising other people’s pages, the same defence that has long protected search engines in Germany. The court disagreed. AI Overviews, it said, do not just show links but make “independent, new and substantive statements” built by evaluating and combining third-party sites, so Google must treat them as its own, including as the provider of an AI system under the EU’s AI Act. “Only it has influence over the AI,” the judges wrote.

Google said at the time it would appeal. That appeal has ended in a settlement, The Decoder reported on Saturday; the terms are not known and neither Google nor the publishers would comment. The upshot is that no higher court has ruled, so the Munich judgment is not binding precedent, but lower courts such as Aachen are already following it.

...

563   Original
kehet
sopuli.xyz

kehet@sopuli.xyz
Joined: 2024-04-24
#2 Posted: · edited   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

For a company earning $403 billion in annual net income, €250,000 is just another budgeted operating expense. Fines should always be based on annual worldwide revenue.

126
Granbo's Holy Hotrod
lemmy.world

uberdroog@lemmy.world
Joined: 2023-07-07
#3 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Why is anyone listening to AI? We were told not to use Wikipedia for reports....wtf

56
Solumbran
lemmy.world
Solumbran@lemmy.world
Joined: 2023-06-14
#4 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Why is anyone listening to AI? We were told not to use Wikipedia for reports....wtf

Because wikipedia was never backed by billionaires owning virtually everything

8
randomname
scribe.disroot.org
randomname@scribe.disroot.org
Joined: 2025-01-29
#5 Posted: · edited   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Why is anyone listening to AI? We were told not to use Wikipedia for reports....wtf

I fully agree, but the thing here is this doctor who was accused of a sex crime he didn't commit. They can't simply throw out a chatbot without a human being to be held accountable. This is one of the major threats of LLMs in my opinion: the danger isn't the LLM itself, but its creator's attempt to reject any human responsibility.

42
Granbo's Holy Hotrod
lemmy.world

uberdroog@lemmy.world
Joined: 2023-07-07
#6 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

I fully agree, but the thing here is this doctor who was accused of a sex crime he didn't commit. They can't simply throw out a chatbot without a human being to be held accountable. This is one of the major threats of LLMs in my opinion: the danger isn't the LLM itself, but its creator's attempt to reject any human responsibility.

Its the same as the SaaS revolution. The accountability is outsourced and everyone can just shrug. Same as police violence. Homeless camp gets disrupted and now there is a flower patch...huh. If no one is responsible, than no one has to be responsible.

15
Sludgehammer
lemmy.world

Sludgehammer@lemmy.world
Joined: 2023-07-03
#7 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

For a company earning $403 billion in annual net income, €250,000 is just another budgeted operating expense. Fines should always be based on annual worldwide revenue.

Eh, when you consider how often LLM's spit out wrong answers it could add up quick.

44
ZapBeebz_
lemmy.world
ZapBeebz_@lemmy.world
Joined: 2023-06-18
#8 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Eh, when you consider how often LLM's spit out wrong answers it could add up quick.

Is that supposed to be a downside? Seems like the consequences of not producing accurate and fully tested products that have the potential for massive negative effects

32
Jiral
lemmy.world
Jiral@lemmy.world
Joined: 2026-06-06
#9 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Is that supposed to be a downside? Seems like the consequences of not producing accurate and fully tested products that have the potential for massive negative effects

No, it is not a "downside", it is why this court ruling matters.

20
calcopiritus
lemmy.world
calcopiritus@lemmy.world
Joined: 2023-09-02
#10 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Eh, when you consider how often LLM's spit out wrong answers it could add up quick.

Make it a recurring subscription instead.

10% of yearly revenue global for making a machine that lies (in a legally-compromising way) once per year. You want a lie per month? That will be 120% of your revenue for that year.

The numbers can be tweaked, but it should not be tolerated. Were I to misbehave and say what Google's "AI" said, I would be in much more trouble than that.

2
Asetru
feddit.org
Asetru@feddit.org
Joined: 2024-06-25
#11 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Eh, when you consider how often LLM's spit out wrong answers it could add up quick.

Now if only everybody had the same funds as that doctor and could simply sue Google.

8
Canaconda
lemmy.ca
Canconda@lemmy.ca
Joined: 2025-07-10
#12 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

The sooner we make companies responsible for their products the better.

Honestly there should be a lot more Brian Thompsons by this point.

50
BillyClark
piefed.social
BillyClark@piefed.social
Joined: 2026-01-21
#13 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Because wikipedia was never backed by billionaires owning virtually everything

If Google was updating Wikipedia and falsely accusing somebody of sex crimes on their Wikipedia pages, I think you could expect that Google would get sued in that situation, as well. It's really about the source of the information.

8
RampantParanoia2365
lemmy.world
RampantParanoia2365@lemmy.world
Joined: 2023-10-02
#14 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

The upshot is that no higher court has ruled, so the Munich judgment is not binding precedent, but lower courts such as Aachen are already following it.

This sentence sure does read like the author is glad the judgement isn't binding, as opposed to lower courts following it.

6
imetators
lemmy.dbzer0.com
imetators@lemmy.dbzer0.com
Joined: 2025-06-10
#15 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

For a company earning $403 billion in annual net income, €250,000 is just another budgeted operating expense. Fines should always be based on annual worldwide revenue.

Uncapped indexed fines is the tastiest thing any country could implement. I wonder why whole world is not running on them.

6
QueenHawlSera
piefed.social
QueenHawlSera@piefed.social
Joined: 2026-07-14
#16 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

The sooner we make companies responsible for their products the better. Honestly there should be a lot more Brian Thompsons by this point.

Fuck anyone who speaks ill of Luigi

19
ZapBeebz_
lemmy.world
ZapBeebz_@lemmy.world
Joined: 2023-06-18
#17 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

No, it is not a "downside", it is why this court ruling matters.

Fines should actually impact companies, not just be another cost of doing business. By imposing a high dollar value fine per violation, regardless of frequency of violation, companies may actually be incentivized to stop breaking the law in order to continue existing

17
MartianSands
sh.itjust.works
MartianSands@sh.itjust.works
Joined: 2023-06-27
#18 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

The upshot is that no higher court has ruled, so the Munich judgment is not binding precedent, but lower courts such as Aachen are already following it. This sentence sure does read like the author is glad the judgement isn't binding, as opposed to lower courts following it.

"upshot" doesn't mean, or even imply, that it's a good thing. It just means something "result" or "consequence"

5
ReluctantlyZen
ani.social
ReluctantlyZen@ani.social
Joined: 2026-01-28
#19 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Why is anyone listening to AI? We were told not to use Wikipedia for reports....wtf

"Let me google that" no longer is actually searching for an actual article sadly (even if only the 1st), but just parroting whatever its LLM vomits out.

6
redditStinksSuperBad
lemmy.world
redditStinksSuperBad@lemmy.world
Joined: 2026-08-02
#20 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Eh, when you consider how often LLM's spit out wrong answers it could add up quick.
[removed by moderator]
-4
FizzyOrange
programming.dev
FizzyOrange@programming.dev
Joined: 2023-09-24
#21 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

For a company earning $403 billion in annual net income, €250,000 is just another budgeted operating expense. Fines should always be based on annual worldwide revenue.

Detention of its managers might have more impact.

44
FizzyOrange
programming.dev
FizzyOrange@programming.dev
Joined: 2023-09-24
#22 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Why is anyone listening to AI? We were told not to use Wikipedia for reports....wtf

We were told not to use Wikipedia for reports

Yeah maybe in 2003.

10
DarkCloud
lemmy.world
DarkCloud@lemmy.world
Joined: 2024-05-19
#23 Posted: · edited   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

ChatGPT can't discuss Brian Hood because he sued them (it shuts down if you bring him up).

We should all have the option to opt out of being discussed by LLM models. In fact, it should probably be an opt-in process. Not being discussed should be the default.

44
Maya S. Trojan
piefed.blahaj.zone

maya_s_trojan@piefed.blahaj.zone
Joined: 2026-10-12
#24 Posted: · edited   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Fines should actually impact companies, not just be another cost of doing business. By imposing a high dollar value fine per violation, regardless of frequency of violation, companies may actually be incentivized to stop breaking the law in order to continue existing

I think the point other people are making is that this can set precedent in German law and allow other people with similar issues to bring suit and win on merit and not have to fight as hard for their case to be considered. It could also end up inspiring similar cases in other countries using similar frameworks, adapted for local laws.

If suddenly Alphabet has thousands people taking to them to court in jurisdictions all over the world, that can indeed hurt them, although it will take longer than injunctions on Google now, I fully agree.

15
ByteJunk
lemmy.world

ByteJunk@lemmy.world
Joined: 2023-07-04
#25 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Fines should actually impact companies, not just be another cost of doing business. By imposing a high dollar value fine per violation, regardless of frequency of violation, companies may actually be incentivized to stop breaking the law in order to continue existing

There's a difference for a one-time fine that covers continued wrong doing over a period of time, and a fine about a specific instance of misbehaviour.

The first don't accrue, and I agree, should be based on revenue so that they serve as actual punishment and serve as future deterrent.

But a €250k that can be slapped by thousands of cases that happen every day (AI providing wrong information) has the potential to be massive as well.

20
Granbo's Holy Hotrod
lemmy.world

uberdroog@lemmy.world
Joined: 2023-07-07
#26 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

We were told not to use Wikipedia for reports Yeah maybe in 2003.

Yeah, pretty much that exact window. Before that it was dont trust ANYTHING on the internet.

12
TachyonTele_Esq
piefed.social
TachyonTele_Esq@piefed.social
Joined: 2026-06-29
#27 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Detention of its managers might have more impact.

That's exactly what needs to be done

4
FizzyOrange
programming.dev
FizzyOrange@programming.dev
Joined: 2023-09-24
#28 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

That's exactly what needs to be done

Well good news then I guess.

3
UltraGiGaGigantic
lemmy.ml

UltraGiGaGigantic@lemmy.ml
Joined: 2024-08-23
#29 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Detention of its managers might have more impact.

In before managers become minimum wage contract gig economy workers. Gotta Weasle word proof it somehow.

14
UltraGiGaGigantic
lemmy.ml

UltraGiGaGigantic@lemmy.ml
Joined: 2024-08-23
#30 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

I fully agree, but the thing here is this doctor who was accused of a sex crime he didn't commit. They can't simply throw out a chatbot without a human being to be held accountable. This is one of the major threats of LLMs in my opinion: the danger isn't the LLM itself, but its creator's attempt to reject any human responsibility.

Deny justice long enough and you shouldn't be surprised when the people extract their own form in the worst way possible.

3
UltraGiGaGigantic
lemmy.ml

UltraGiGaGigantic@lemmy.ml
Joined: 2024-08-23
#31 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Its the same as the SaaS revolution. The accountability is outsourced and everyone can just shrug. Same as police violence. Homeless camp gets disrupted and now there is a flower patch...huh. If no one is responsible, than no one has to be responsible.

Maybe I am immune to responsibility and I just haven't tried it out yet.

1
ElDuderino
lemmus.org
ElDuderino@lemmus.org
Joined: 2026-08-13
#32 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Yeah, pretty much that exact window. Before that it was dont trust ANYTHING on the internet.

Don't trust anything on the internet should have really never gone away.

11
Sludgehammer
lemmy.world

Sludgehammer@lemmy.world
Joined: 2023-07-03
#33 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

[removed by moderator]

?

I fail to see how you got the idea I support LLMs or their companies from my post.

2
who
feddit.org
who@feddit.org
Joined: 2025-04-02
#34 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

FTFY: Google pinned another man’s sex crime on a doctor, and a German court says Google must answer for it.

12
shalafi
lemmy.world

shalafi@lemmy.world
Joined: 2023-07-22
#35 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

ChatGPT can't discuss Brian Hood because he sued them (it shuts down if you bring him up). We should all have the option to opt out of being discussed by LLM models. In fact, it should probably be an opt-in process. Not being discussed should be the default.

Forgot about that! Just tried it.

23
shalafi
lemmy.world

shalafi@lemmy.world
Joined: 2023-07-22
#36 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Uncapped indexed fines is the tastiest thing any country could implement. I wonder why whole world is not running on them.

Sounds great, but shitloads of tiny laws have to be changed and shitloads of precedent overthrown, not as easy as it sounds I'm sure.

1
halcyoncmdr
piefed.social
halcyoncmdr@piefed.social
Joined: 2026-01-15
#37 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Eh, when you consider how often LLM's spit out wrong answers it could add up quick.

Sounds like they should be trying to prevent obviously foreseeable issues then.

8
pycorax
lemmy.world
pycorax@lemmy.world
Joined: 2023-08-09
#38 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Detention of its managers might have more impact.

Unless we consider every manager above the manager directly responsible all the way to the top, just sounds like a way for them to scapegoat someone and get away scotfree.

13
refalo
programming.dev
refalo@programming.dev
Joined: 2024-04-13
#39 Posted: · edited   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

I fully agree, but the thing here is this doctor who was accused of a sex crime he didn't commit. They can't simply throw out a chatbot without a human being to be held accountable. This is one of the major threats of LLMs in my opinion: the danger isn't the LLM itself, but its creator's attempt to reject any human responsibility.

I don't consider that even a threat of LLMs, just humans being human.

Just like we don't blame video games for violence.

-3
Ephera
lemmy.ml
Ephera@lemmy.ml
Joined: 2020-05-31
#40 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Why is anyone listening to AI? We were told not to use Wikipedia for reports....wtf

I mean, I have to say, recently Qwant introduced default-on AI responses and one reason I hate it, is that it shows up where they also show excerpts from Wikipedia etc.. It has happened to me that I was reading the generated text thinking it's an excerpt.

I would not be surprised, if tons of users never realize that it's generated.

9
Ephera
lemmy.ml
Ephera@lemmy.ml
Joined: 2020-05-31
#41 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Should have been very obvious a long time ago that you can't just publish wrong information, no matter how new and exciting your technology is...

25
14th_cylon
lemmy.zip
14th_cylon@lemmy.zip
Joined: 2025-06-29
#42 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

ChatGPT can't discuss Brian Hood because he sued them (it shuts down if you bring him up). We should all have the option to opt out of being discussed by LLM models. In fact, it should probably be an opt-in process. Not being discussed should be the default.

then you might as well demand they stop stealing people's work and reselling it for profit. where would that stop?

19
Sludgehammer
lemmy.world

Sludgehammer@lemmy.world
Joined: 2023-07-03
#43 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Sounds like they should be trying to prevent obviously foreseeable issues then.

Well, unless they want to pay a quarter million Euro every time their chatbot lies about someone, yeah.

3
cafuneandchill
lemmy.world
cafuneandchill@lemmy.world
Joined: 2024-01-26
#44 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

In 2026, an AI on crack has accused a man of a crime he didn't commit

11
EliteCloneMike
lemmy.zip
EliteCloneMike@lemmy.zip
Joined: 2026-04-29
#45 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

These kinds of AI output should not be protected by anything like Section 230. It is their own words and they should not get a pass for what essentially amounts to libel. Google needs to be reeled in. The fines have been too small and the damage has been too great. This isn’t even the first time something like this has happened. They have made up claims like this against plenty of other people. Claiming that they get a pass because it is AI is just using the LLM, their own LLM, as a scapegoat.

Other examples
https://www.theguardian.com/music/2026/may/05/canadian-ashley-macisaac-fiddler-musician-singer-songwriter-sues-google-ai-sex-offender-ntwnfb

https://londondaily.com/robby-starbuck-sues-google-after-its-ai-made-up-totally-fake-rape-and-crypto-fraud-claims-against-him-and-supported-it-by

Slightly different but related
https://www.eff.org/deeplinks/2022/08/googles-scans-private-photos-led-false-accusations-child-abuse

17
NotMyOldRedditName
lemmy.world
NotMyOldRedditName@lemmy.world
Joined: 2023-08-24
#46 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Forgot about that! Just tried it.

I wonder if that'll have downstream effects like you ask it to name everyone who went to his highschool and it errors out since his name comes up.

3
BlindPenguin
piefed.social
BlindPenguin@piefed.social
Joined: 2026-09-04
#47 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Unless we consider every manager above the manager directly responsible all the way to the top, just sounds like a way for them to scapegoat someone and get away scotfree.

There is hope, that they might run out of managers, though /s

2
Avicenna
programming.dev
Avicenna@programming.dev
Joined: 2025-11-07
#48 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

When there is a legal issue "it is only an advanced search engine". When it is time for PR "It is already more intelligent than most humans"

11
Avicenna
programming.dev
Avicenna@programming.dev
Joined: 2025-11-07
#49 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

For a company earning $403 billion in annual net income, €250,000 is just another budgeted operating expense. Fines should always be based on annual worldwide revenue.

Yeah their CEOs likely spend this amount of money in a high profile client dinner

2
Fmstrat
lemmy.world
Fmstrat@lemmy.world
Joined: 2024-04-27
#50 Posted:   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

For a company earning $403 billion in annual net income, €250,000 is just another budgeted operating expense. Fines should always be based on annual worldwide revenue.

That appeal has ended in a settlement.

The fines, if even paid, were probably less than the settlement. That being said, the best case would have been going to court for precedence, but you can't blame someone for cashing out if they've been a victim.

1
Kissaki
programming.dev

Kissaki@programming.dev
Joined: 2023-06-11
#51 Posted: · edited   Subject: Re: Google’s AI pinned another man’s sex crime on a doctor, a... Quote Reply

Forgot about that! Just tried it.

who is brian hood

Depending on the context, Brian Hood most commonly refers to one of two notable figures: […]

It tells me about two Brian Hoods. I assume one of them is the supposed denied one?

Or it is selective about what information it gives about them.

1
Post Reply Page 1   Next