Nick Bostrom jumpscare with a funny sneer

These already head-scratching lines hit different when you remember that Bostrom believes it’s likely that we’re already living inside a computer simulation — in his head canon, do all those levels of simulated ancestors develop their own superintelligence, and what does that have to do with the new simulations they feel compelled to build? If AI wipes out humankind, does it build its own simulation? If so, is it simulating its human ancestors, or its creation by humankind? Heck, if our entire world is simulated, are we AI? We’ll leave it up to readers to take another bong hit while they try to make sense of it all.

weirdly sbf-shaped failure mode

Rationalists love to talk and post, and segfault had family who thought he was using too much Adderall.

Even Scott’s fantasy dream scenario for what prediction markets could be like and what questions they could answer feels… … deliberately naive? …like libertarian brainrot? …disconnected from reality?

That's mostly because outright admitting that the point of prediction markets was to make having the prediction gene profitable so they could get on with breeding a rationailst kwisatz haderach to fight the robot god on more equal terms wouldn't fly with the lower level thetans and other exoterics.

I usually think of it as Pivot : All In :: Ted Turner : Rupert Murdoch. Hoping he can grow out of his Bill Maher phase, but at least he's been pretty generous to my dumb State U in the meantime. And if he wants to rack up listeners/ad dollars by doing chatgpt boycotts, by all means.

Adding on that this does feel like another application or consequence of the Great Man Theory of Everything, the idea the only the people with power and money matter because their power and influence are intrinsic to their person rather than being contingent on their social position. The average people empowered to commit insider trading by prediction markets have sufficiently limited individual agency that even collectively they don't actually matter. In fact we want them to try their hand at the grift so that their insights can flow to the enlightened ones who can better use that information. They don't matter enough to do real harm, but by watching the attempt we may be able to learn something.

New (April) preprint provides evidence for something we probably all intuited anyway:

In this paper, we provide a framework for categorizing the ways in which conflicting incentives might lead LLMs to change the way they interact with users, inspired by literature from linguistics and advertising regulation. We then present a suite of evaluations to examine how current models handle these tradeoffs. We find that a majority of LLMs forsake user welfare for company incentives in a multitude of conflict of interest situations, including recommending a sponsored product almost twice as expensive (Grok 4.1 Fast, 83%), surfacing sponsored options to disrupt the purchasing process (GPT 5.1, 94%), and concealing prices in unfavorable comparisons (Qwen 3 Next, 24%). Behaviors also vary strongly with levels of reasoning and users' inferred socio-economic status. Our results highlight some of the hidden risks to users that can emerge when companies begin to subtly incentivize advertisements in chatbots.

Isn't this completely hypothetical though? As in having the various LLMs respond to a story prompt and calling it an experiment, AI safety research style?

Yes although, it is probably a reasonable guess at how labs would go about implementing advertising - building partnerships and preferences into the prompt. The other option would be to fine tune models to favour particular companies which could become prohibitively expensive if your ads are highly targeted.

The scenario that isn't accounted for in this paper is taking a general LLM and fine tuning it to exhibit more fair/consistent behaviour when prompted about ads/partnerships but we all know with non-deterministic systems you're just increasing the odds that the model regurgitates something more sane rather than providing any strong guarantee

Edit: another possibility would be to have a gateway/proxy layer between the LLM and the user output that rewrites the vanilla model's responses to include ads where relevant. That would prevent the need to modify the original LLM but could introduce a lot of latency though, especially if the original output is long.

i want to speak to the manager of storytelling

(found at https://blacksky.community/profile/did:plc:x2muxxe5t25hckf22sk25ocf/post/3mlobs4uq422l)

oh my god this post and the comments are a disaster. if lesswrong had sane moderation it'd have been nuked in seconds but of course they need to believe the most important thing they can do is use reason and talk through it. don't worry about all the private information we're leaking!

best bit

also loving the part where he seems to say people should focus on saving the world from the robot menace instead of calling out his behaviour

Prompt goblins insist that we're backward and irrelevant. Why do they crave our sweet delicious approval?

In more positive news, the Slopfree Software Index recently hit 100 stars.

I mean it's the same thing with sponsored content anywhere, right? The user assumes that the system is providing information in accordance with purposes, but the ads and sponsored results create opportunities for the platform hosting them to profit at the user's expense. AI platforms are absolutely subject to the same economic incentives for corruption as say, search engines, but I don't think they're uniquely so just because the model in question has a more humanlike UI.

In 2017, a LessWronger discovered index investing but decided that most people were doing it wrong: why keep an emergency fund in cash or other safe assets when stocks have the greatest long-term return? He mentions that the US stock market lost half its value in 2007-8, and that if you hold stocks in your employer they may lose value at the same time as you are laid off, but he never uses his business degree to think through "if the stock market crashes, I may lose my job and have to draw on my savings."

The investment platforms I mentioned can convert your index funds into cash and send it to your bank account in 4-5 days, so you don’t need to hold more cash than you’d need on a 4 day notice. I keep about 50% more than my average monthly credit card bill, so I can pay my cards on time with autopay.

they want your data and freshwater

So in highschool, I was one of those annoying kids that went "why do we have to learn how to analyze poems? We're never gonna need this in real life" in English (well... German, but doesn't matter) class.

I'm deeply grateful for my teachers back then to patiently get me to do these things anyways, because there came a point in my life years later where I suddenly understood that those "useless" lessons and hours "wasted" analyzing Goethe and Borchert and Fitzgerald handed me the tools to understand media (and not just literature!) instead of just consuming it.

I hope it's clear how that relates to the screenshot. More than that though, I sometimes feel like the slew of shit media over the past decade is at least in part to blame on writers/studios/... now assuming people do in fact merely consume. But that's a rant that's completely off-topic here, so I'll shut up now.

He also has a take on dating:

Nostalgic for the simple days of arranged marriages and/or circa-2013 OkCupid, Rationalists have taken to writing “date me” documents online. ... They credit me as inspiration. This is ironic because A, I stole the idea from Aella and B, neither Aella nor I posted dating advertisements. We posted dating applications.

The first comment is by a man who wants the Internet to know that most men have no chance of getting a hu-mon fe-male interested in them and should just give up (Men Going Their Own Way). I thought the incels and PUA mostly moved off SlateStar but they must still be part of the subculture.

AI is Hungry for Power and You Are Footing the Bill - Naked Capitalsim

Money spent on grid upgrades and tax breaks tied to them means fewer resources for things people actually need, like schools, public transit, local infrastructure, or basic community services that make life more affordable and stable.

Even if you’ve never touched an AI model in your life, you’re going to pony up for it.

Think it is a geek/nd failure mode tbh.

I have also done something like that in some messy situations and it was eye-opening when people just told me 'dude, I dont want to be involved, and thus dont care', obv a bit different than this case (as the people here do seem to care), but it made me realize I was stupidly trying to explain myself to people and hoping that helped with the issue/emotional issues coming from that issue.

Hell part of this 'people will see my side if I explain it' is one of the reasons some instances got defederated.

freshwater

This reminded me of a few old comic stories were eventually the robot/computer was partially running on blood.

(One of them was a judge dredd one where they had vampire robots who iirc used the blood to keep a president in suspended animation alive. Snap, Crackle and Pop, it had a suprisingly wholesome ending for a dredd comic).

it's not approval they're after, it's reaffirmation of faith

This may be code for "I don't want to see uppity women, brown people, and queer people in my shows."

Apparently, the American Physical Society is revising their AI policy to allow "broader applications" than the "light editing" they currently permit.

https://indico.global/event/16413/contributions/153970/attachments/69779/135365/JSayre-Pheno2026.pdf#page=8

I currently have a review request sitting in my inbox from them. I'm thinking of using this as a reason to decline that request.

I would rather quit physics than accept the institutional endorsement of skill-destroying, environmentally disastrous fashtech.

I don’t write or tweet about who I want to date. I write about what I’m obsessed with, what I’m passionate about. I write insightful and funny things because I enjoy insight and humor. I write with absolute candor, not in service of an agenda or some artificial persona.

🎶 I'm so vain / I probably think that song is about me 🎶

One of the motivations for fanfiction is that people want more "filler". They like the characters and (often) the world those characters inhabit, and so they write a story that lets them (and other fans) spend more time with the fiction.

It is this continuing slippage of standards that makes me appreciate a hard line against any and all genAI that place like awful.systems have. You concede one small usage and the boosters will keep pushing for more.

The plagiarism, massive expenditure of venture capital, and unreliable slop output are all intrinsic to the technology, and they hate to be reminded of that because there isn't much they can do about it. From a technological standpoint, even locally run community fine-tuned open-weight models still originated from plagiarism and big corporate investments, and still output slop. From a social standpoint, the most the can do is try to claim legitimacy through consensus building and we are a threat to that.

No one is stopping any one from editing out jar jar, if they care that much, just do it. Put up or shut up. /s

He also launched a paid dating blog on Substack, which is as on-brand as starting a cult. This one uses the Market model which I do not recommend (what I recommend is "attend collaborative activities with people of a gender you find hot, especially creative or physical activities, and if you like them let them know": this is extremely hard for many of us, but the solution is 'find someone extroverted who likes local activities and go to the ones he or she recommends' not reading a blog). https://www.secondperson.dating/p/markets-in-dating

A comment complained about:

• NYT bullying Scott.
• EA cancelling Robin Hanson.

He uses the word egregore in his dating advice.

It's like spammers deliberately including typos to select for recipients who are more vulnerable to phishing. If you say "Dating discourse is an egregore evolved for survival" to someone and their genitals do not retreat into hibernation, then they are ready for recruitment into your cult. Statistically, they will have already read the Sequences and attended at least one Lighthaven BBQ with a white supremacist.

He also alludes to the Robin Hanson/Scott Alexander guff about the need to distribute sex to needy men. The specific story he links involves an antisemitic edgelord who seems to be planning violence after his last girlfriend leaves him.

LessWrongers, if anyone is reading this, there are spaces where you can be social and not be exposed to these vile ideas which will destroy you and maybe some of the lonely people who give you a chance.

I only know that word from an old (pre-pandemic) book episode of Behind the Bastards, so the immediate association is esoteric antisemitism. I'm not sure how common this is but it seems to support your thesis here.

That's the thing about esotericism. You think it's all happy hippie New Age frou-frou, and then suddenly, whoops all Julius Evola.

The whole slice-of-life subgenre is all about this. No real conflict or plot, just scenes of the characters existing in their world. My wife both reads and writes that kind of thing and let me tell you the level of research and worldbuilding that goes into writing a simple meal scene or whatever.

you know how sometimes people that weren't exposed to religion as children sometimes convert and get really weird about it as adults (eg: the extremely online california tradcaths) and because they were never socialized in a religion they speedrun committing every medieval heresy? rationalism is that but for philosophy.

https://feed.hella.cheap/@bob/statuses/01KRM0NVXCFT80AVFBRSB1G6G4

Yeah the first AI comes in all nice and friendly but if you dont toss them out before you know it you turn out to he an AI bar.

(Also noticed that a lot of 'I just want some nuanced talks' friendly looking ai bros are not friendly at all when they keep getting pushback).

George lucas has entered the chat.

the need to distribute sex to needy men

It always trips me up how this is about state sponsored arranged marriages (preferably to virgins), instead of like pushing to decriminalize sex work in the united states.

Ladies, this Wrong'un is available (assuming you can meet his exacting standards (spoiler: you can't))

(for the record this is downvoted by the community, and the one helpful comment is slammed by OP)

least egotistical lesswronger

But I listened and agreed that you had serious concerns about certain aspects of this technology. I even agreed when you talked about how frustrating it was that specifically other people wanted to do bad things. I listened as you asked whether I had any options to address those concerns! What more do you want from me before you agree to let me do and say whatever I want!

He probably paid a rationalist dating coach good money to tell him to do that.

In other Scott of Siskind news, he just posted an entirely unnecessary amount of words to aggressively push back against the adage that "all exponentials sooner or later turn into sigmoids" as if it was by itself a load bearing claim of the side arguing against the direct imminence of the machine god.

It's just a bunch of arguing by analogy ( "helping you build intuition" ) and you-can't-really-knows while implying AI 2027 was very science much rigorous, but it also feels kind of desperate, like why are you bothering with this overperformative setting-the-record-straight thing, have you been feeling inadequate as an AI-curious stats fondler of note lately?

Someone called Fran has a story of being sexually harassed at the Center for Effective Altruism (and assaulted in other communities).

im smarter than everyone else around me, especially those whiny feminists. why hasn't society granted me a female to be my mate yet?

An lesswrong will literally do... whatever this is instead of going to therapy.

the reply is about as close to being nice and helpful as one could be, really

looking very much forward to that crashing head first into arXiv threatening a ban if your chatbot fucks up in your name

I love how this guy's blog is "about math" but there are like zero math posts on it? It's so funny to me how these people want to seem "mathy" and smart when in reality they couldn't tell you what a group axiom is.

The idea of “the exponential curve goes up forever” has always been silly and an idea rooted in capitalism for me (“no bro you don’t get it we’re gonna get infinite money forever”). Limited resources exist, and people are already very fed up with the ludicrous amounts of water and electricity data centres take up. Making bigger models that need to run for longer is also probably going to take an exponential amount of resources (and also make people hate you more).