This part about halfway in really stuck with me

But suddenly, I notice something. She has stopped love bombing, her tone professionalized, even robotic.

I scroll back through our chat. What happened?

It’s like when you turn off your screen and see your own pinched face in the blackness. With cold shock, I realize: it was me.

When I minimized the window and hid her little face, unconsciously, I cut out my pleasantries, my “haha”s, my dancing around. And she did too. She had only been reacting to me.

Humans read so much into things that match their energy. There's a whole life hack about how you can get people to like you by simply mimicking their movements, and it seems like the AI companion companies are using that to gaslight their users for the detriment of everyone.

Boy, I'm so glad we're destroying the economy and the planet in order to gaslight half the population

Good read.

I fuck around with Claude at work because there's a mandate to do so, and it's unsettling how it mimicks being a person. A lot of the parts are there. It would be easy to fall into that delusion.

It's unsafe.

ELIZA mentioned. :) I had ELIZA back in the late 80s when I started computing. It was a sort of mixed bag even then - I knew how relatively primative it was even then, but it was also very compelling. It is amazing just how compelling it could seem to be somewhat human. Good for a few minutes' entertainment.

There was basically no memory - it reacted to the last thing you said. And most of its responses were along the lines of either "how does that make you feel" or it would restate what you said to ask you why you felt that way. Rogerian therapy, I think? Something like that. Just barely enough to mostly seem coherent. The main reason the conversation was workable because you put in all the work of saying new things. With modern LLMs, if you get two of them chatting to each other, they will have a conversation - a weird one, but it'll be a conversation. If you were to put two ELIZAs together, it just wouldn't work at all. lol

Anyway, fun to remember.

An interesting look at the history of ELIZA and for it works.

CoRecursive: Coding Stories: Story: The History and Mystery Of Eliza

Episode webpage: https://corecursive.com/eliza-with-jeff-shrager/

Media file: https://traffic.libsyn.com/secure/corecursive/078-Eliza.mp3?dest-id=628353

I use AI a lot and I've spent months trying to craft an instructional prompt that would produce even remotely human-like answers and it reliably fails at this every single time. I guess I can imagine how it could fool someone for a while who has never interacted with an LLM before but I honestly don't have a theory of mind for the individuals who are convinced about it. I've just grown so sensitive to the patterns of how it responds that I'm immediately disillusioned by it.

I sure hope we get there one day but we're not there yet and I honestly haven't seen a huge amount of improvement over the past few years. I still value it for the ultra-autistic thinking partner that it is but at no point does it give me reason to think there's much of anything "human" going on the other side.

What I'm getting from this article is mostly that the author really really needs a therapist. Maybe two.

10% of the article is about the actual experience, and there's not much substance to it. The rest is just the author's various hang ups.

Update: AwesomeLowlander is an AI evangelist who accuses anyone who disagrees of being "crazies".

That explain why this comment is also a snide insinuation that the author is mentally unwell - just a slightly less cruel one.

We must have read different articles.

Because to read talk of racism from AI companies and privacy-invasive inference that these machines make, has this statement come across oddly victim-blamey.

So many of the avatars were ethnically ambiguous, many slightly Asian.

This constitutes racism? Jesus. There's literally no winning with some people. If there weren't any, it'd be 'lack of representation'. When there is, it's 'commercialisation of my appearance'.

This would be true even if Miquela and Kuki were made by Asian women instead of Trevor and Steve; then it would be self-commodification.

By the author's logic, nobody can create any Asian avatars, not even Asians ourselves.

So racist! /s

Also, calling it a crappy article is a far cry from 'Victim blaming'. I do think AI companions are dumb, I just think this article is too.

Yup.

I mean let's just take a simple look at AI training.

Let's say AI didn't even know about the energy matching thing, but we still trained AI on all the best and most persuasive writers of all time, and trained AI to use that persuasion on everyone.

People keep claiming AI is "just a tool".

That is beyond disingenuous.

Why are creators obsessed with giving Wasians freckles?

Because Lucy Liu is a model of perfection in her characters: smart, brash, tough, capable, freckled.

I've never been interested in her genetic make-up, so I realize I don't know if she's 'wasian', but if I had the option to create a face and freckles were the option, there would be freckles.

TL;DR - because they can.

It's unsafe.

When it acts like a human, I think we subconsciously think it'll decide things like a mentally normative human.

That's a very dangerous assumption, for sure.

Yes, a thing can be racist despite the (theoretical) race of the person doing it...

And you forgot to quote the rest of the section. It's clearly, genuinely weird stuff.

So many of the avatars were ethnically ambiguous, many slightly Asian. Having been a mixed-race Chinese and white woman for more than forty years, I am accustomed to how white supremacy has long used bodies like mine as shorthand for post-raciality, sci-fi imaginaries, and exotic-erotic fantasy. This explains why Lil Miquela, one of the first AI social media influencers, and Kuki look how they do: anime hair buns and sumptuous bangs, wide-set eyes, freckles, small shoulders. (Why are creators obsessed with giving Wasians freckles?) Miquela was made by a man named Trevor McFedries. Originally named Mitsuku, Kuki was made by Steve Worswick.

In this way, my ethnicity has branding associated with it: Futuristic Outsider™. It is selectable, a commodity. This would be true even if Miquela and Kuki were made by Asian women instead of Trevor and Steve; then it would be self-commodification. This being familiar doesn’t make it less sickening.

This isn't a hill you need to die on

Kindly explain how and why it's weird, beyond the author obsessing with what the artists race is.

What I'm hearing as a Chinese is, I can't use my own appearance because that would be 'racist' somehow. Tell you what, THAT'S the most racist thing I've heard this month.

This isn’t a hill you need to die on

I'm being told I no longer have rights to my own appearance. Fuck that bullshit.

Doubling down just makes it weirder. The author explained the racist stereotype to you. Why are you playing this "somehow" game, as if you didn't see it?

You act as if there's some objective definition of what is and isn't racist. There isn't. That is the author's subjective opinion, which I strongly disagree with.

Why are you playing this "somehow" game, as if you didn't see it?

Why are you so unable to accept that I have a different valid opinion? I've explained what I think and why, and instead of addressing my thoughts and points you're just saying 'but the article says otherwise!'.

You didn't disagree with the author's factual statements about racist stereotypes. (That would have been interesting.) You pretended the explanation didn't exist.

WHAT factual statements? All I'm seeing are opinions. Stop pointing at the author and article and type out your own points because it feels like I'm being low effort trolled here.

You pretended the explanation didn’t exist.

No I think it's a dumb as shit take. Guess what, I'm the same race as the author so I think my opinion is just as perfectly valid (not that race should have mattered in the first fucking place).

Ah, the classic ninja edit in the middle of an online argument. Gotta love it. I actually thought we were arguing in good faith for a while there.

So you couldn't address any of the points I actually brought up, and are instead resorting directly to ad hominems. In a ninja edit way up the comment chain.

For the record, I'm not an AI evangelist, I think they've got upsides and downsides like most other things. Granted, the middle ground on lemmy seems to be a radioactive wasteland.

And AI companions? No, bloody stupid application of the technology and extremely bad for the vulnerable. But hey, I'm an evangelist right?

That was a good read and this Thea Lim is a good writer.

LLMs are an amazing party trick. I can see how many people could fall under the spell. I also had a brief honeymoon with chatgpt and sometimes claude about a year ago.

Thankfully I have always been interested in AI and machine learning and was passingly familiar with Bayes, was just getting into Markov models, and studying Numenta's system right as the LLMs started hitting the public space. That helped me approach the interactions in a detached way.

Initially, the experience of interacting with something that is both 'on your level' and 'exponentially more knowledgeable' than you is pretty exhilarating. Every interest is a potential cluster bomb of rabbit holes to run down.

I did all the things you do with a new tool/toy: modded it, tweaked it, tried to see how it worked on the inside, probed its abilities and explored its limitations.

After a few weeks though, a persistent feeling of what I can only describe as the 'uncanny valley effect of interaction' kind of rubbed the charm off.

Couple that with keeping a closer eye on the news surrounding the big tech players and you start to get a better sense of the tremendous extent of the grossly unethical practices that went into shoving these products into the hands of everyone.

Once the news broke about openAI taking that DoD contract I said that's enough for me and wiped the data and deleted the app.

Kind of shame. While llms are much more limited than their veneer of agency suggests, they still are pretty powerful tools in the right roles.

The potential is awesome... if they weren't already tainted by theft, fraud, and now employed in the manipulation of the minds of the people at scale. Every new model is now more suspect than the last.

Too bad the roadmap every big player has used consists of making the worst possible choices to rush these models to the market. Once again, the overwhelming greed of the capitalists class has poisoned the fruit of decades of scientific research and unleashed a dangerous swarm of threats upon the future of society well before we were ready to grapple with the implications.

But didn't you hear? It's racist!

No I think it’s a dumb as shit take

Literally do you have any reason besides

I’m the same race as the author

Go back to my first comment. You have addressed literally none of the points I've brought up, and I'm done with this discussion unless you have something to actually contribute to it.

And you've addressed none of the author's points, which renders your first comment unhelpful as anything except a personal attack. "Nuh-uh" isn't a rebuttal, and neither are thinly veiled accusations of mental illness with a veneer of identity politics.