RetroFed Forum
Your home away from modern life.
AI tool used in asylum case reviews failing accuracy tests
Post Reply
AuthorMessage
CombatWombat feddit.online CombatWombat@feddit.online
Joined: 2025-09-09
Quote

Asylum seekers are not informed that caseworkers use an AI tool to summarise their interviews used by decision makers

24   Original
CombatWombat feddit.online CombatWombat@feddit.online
Joined: 2025-09-09
#2 · Quote Reply

That's a low bar, and it still fails 22% of the time

In the first stage of the review, 139 summaries passed, 19 were flagged up for further review and 45 (22%) failed. In the second stage review, 56 (15%) of the sample summaries failed.

However, AI-generated summaries that omit material facts about the case can still pass if the rest of the content is accurate. This is because the relevance of content included in the summaries is not listed as one of the grounds for assessing failure.

5
dumnezero piefed.social dumnezero@piefed.social
Joined: 2024-11-27
#3 · Quote Reply

slopocracy

1
Post Reply
Back to top