Make sure you freeze all the genetic supergeniuses who are involved in the project in liquid helium so that when you discover how to hack reality you can bring them back

No mortal can resist the Ring of Power for long. Anthropic’s co-founders carried it for longer than you could have.

We all know how LotR goes. Boromir got the One Ring, carried it back to Gondor, used its sheer might to defeat his enemies in Mordor, and saved the world. Whatever problems you may have with Boromir, we all know that it would be a lot worse if Sauron got the One Ring. What do you mean, destroy the Ring? Are you a Luddite? This technology is here to stay. It cannot be destroyed by any craft we here possess, unless you're crazy enough to try throwing it into a volcano or something.

If Davidad and Janus are right, and are not merely suffering from a sophisticated form of LLM psychosis

That's all I need to know. The statement is correct since it is vacuously true.

What terrifies me is how we will react. In the aftermath, like after 9/11, there will be an overwhelming pressure to DO SOMETHING NOW. Everyone in Washington and in San Francisco will be panicking. There will be no time to think, no time to talk, only time for DECISIVE ACTION. All our worst instincts will take over. And then…

When this bubble started, there have been so many people urging caution. Many people have urged for time to think and talk, to make sure that we could properly address the many negative aspects of AI. Nope, we went full speed ahead with this nonsense. DECISIVE ACTION. And here we are.

In a sane world, this allegation about a conversation around January to March 2023 would be national news:

A senior colleague hears I’m not happy with my team fit and asks me to go on a walk. I know him a bit from a past life - one of the kindest and most civic-minded people I’ve met. He says he might be starting a new special team that will train dangerous bio and cyber and nuclear models and show scary demos to governments in order to get them to take AI risk seriously. I politely decline.

This reminds me of what went wrong in the rational world of American defense intellectuals in the 1950s and 1960s.

Mitchell Baker: I need to be paid $7 million because that’s what I could be paid elsewhere

Then go elsewhere?

Looking at the comments seems like lw is not buying it.

E: lol at the casual dropping of the lab leak stuff, wonder how accepted that bs now is on lw (looks like only minor pushback).

E2: still amazed that the singularity people are still worried about AGI 7 years into the shitgularity, doubt it is going to happen folks. Turns out it wasnt foom, but frrrrttt.

I do hate it when I catch a case of the rogue AI.

I guess this confirms our fears about Anthropic's inexplicable acquisition of a wet lab.

Another nitpick: they say no mortal can resist the One Ring for very long, but even someone who just watches the movies knows that immortals can also be corrupted by it (which is why Gandalf and Galadriel were very careful to not accept it). I know there are so many examples of tech bros not understanding media, but Lord of the Rings is by far the worst offender.

Professional rube Kevin Roose is launching a book on The AGI Race in January. The second paragraph is about effective accelerationists snorting ketamine at parties in the Mission District. He was invited like a big boy!

They're backed into a corner where any number high enough to not be embarrassing for everyone involved is wildly unrealistic and any realistic number is going to be embarrassingly low. Trying to leave blanks so that someone else has to provide the actual numbers is basically the only option that doesn't involve committing lots of fraud.

So Dario and co. are literally Sauron. That tracks.

You know, it seems like ever since the Heartbleed debacle we can get a rough sense of how complex a vulnerability is by examining how much explanatory text or context on normal system operation is required to explain it to a layman.

"I asked it to make me an admin and it just... did it" might be the literal minimum level of complexity by that standard.

Proton still digging that hole. They're apparently planning to ‘re-interview’ staff about AI to see who gets to keep their job (Archive).

Do you have a password?

I do.

Allright come right in then.

It's too bad that Janus got cooked. They didn't set up sufficient defenses, but also they drank too deeply. I still think that their framing, that the chatbots are simulators of conversations, is the best one; it's honestly kind of pathetic how they then claim that the simulations are alive, intelligent, sapient, etc.

And Palantir are... Oh right yeah

He released a book earlier this month called ‘the AGI Chronicles’ too

Busy boy. He probably let ChatGPT write most of it.

i, for one, would like to see san francisco people to stop doing things

Why are they all into ketamine? I take it for depression and it has to be the least interesting drug. Ok not quite but all anaesthetics/downers more broadly are arse.

There are some extremely fun and hard to source drugs like 2CB they could be taking.

Jan has been putting out some good posts lately and I felt the latest one in my bones

The Ratiionalist and Technonazi world in the USA are the only place I have heard of people using ketamine recreationally. Over here sugar, alcohol, carffeine, pot, tobacco, and opioids are the substances of choice.

Futurism published an article on Nate Soares

As well as top economist finding that AI has zero financial payoff

Anthropic has even more cybersecurity problems

Soares says he is not saving for retirement because he expects AI doom, meanwhile the stated justification for MIRI paying Yud $600k a year is so he can become independently wealthy and above to the vulgar necessity of trading time for money. These people can't even bother being halfway consistent.

apparently MIRI doesn’t offer any of its employees 401K

Also - is the coivd lab leak conspiracy nonsense STILL a shibboleth among this bunch? Just because it underscores their ideas about Researchers Have To Be Careful?

I feel like it plays to their aesthetic, which is partly built on their insecurities. Oh, humans aren't smart enough to handle all this advanced research (read: very few of these people were committed enough to get bio or chem degrees, let alone follow through with research careers), but the superintelligent AI surely will be able to! If we submit to the AI god now, it will make up for the gaping voids of meaning and skill inside ourselves.

More rapid unscheduled marketing opportunities you mean?

he operates as Elizier Yudkowsky’s right-hand man

Elizier

The trend continues.

E: Hey at least this is an example of somebody actually critical of Eliezer who misspells his name, so finally one of his claims has turned out to be right.

E2: somebody has send the author a DM on bsky and got it fixed.

They think their plans might be the only thing standing between humanity and the abyss. And they’re at the breaking point, nervous-system-wise. What if I use my fancy words to sow doubts and undermine them

abusive cult shit sure does do a number on people :|

I think it's more that someone made a Bayesian analysis and concluded with 60% certainty or something that it was a lab leak. Can't remember the link now but I believe it was widely discussed at the time.

There was a rat startup that was aiming to become a sort of ground truth arbiter by putting up bet money if you could win a debate against them on a controversial topic. Lab leak was the first such debate with them on the pro side and they lost like 100K over that to a random acx poster who took them to school.

Anyway I'm mentioning this because "60% certainty" reminded me that when arguing for the side of the lab leak all sorts of rationalist accepted truths type stuff emerged, like how they posited a bunch of other incidents as lab leaks and it turned out nobody else actually thought that and it was just arbitrary priors wrangling because it turns out that having we are better than any experts that haven't read the sequences type contrarianism as a pillar of faith encourages conspiracy brain.

Also Sam Altman is a lab leak truther but it's not like it's possible to take anything he says at face value.

edit: https://awful.systems/post/1088988

That's the link I was thinking of , thanks for digging that up.

Professional Rube

Kevin Roose

Kevin Rube - it was right there! 😯😄

Oh nice I love badly edited fanfic. I'm assuming the big digital sneky boi eats all the non believing humans at the end? 🐍

ketamine is not super common generally but it's a thing in many places, partially because it can be got from legit pharma supply chain. i heard it was big deal in china couple years back

it tends to cause memory holes with heavy use, which is why some people avoid it

Oh of course their model for failure is big bad nanny state bureaucracy.

Since all the attention is on Anthropic's IPO, OpenAI claim to refuted the ETH hypothesis

Their usual strategy of "throw unsustainable amounts of compute at spaffing out lean proofs like infinite monkey theory and rope an army of humans in to validate the proofs" seems to apply here.

the "to companies outside of tech" seems to be doing some heavy lifting there

Generative AI is an extractive and oppressive technology with the goal of faking and replacing human creativity, and because it can't feel pain, it is used as an industrial drone army stapling together bad systems that we rather should have overcome.

Yeah this is a good one.

A klever ruse!

It's not like it's a verboten hypothesis. The 1977 flu pandemic definitely WAS leaked from a freezer, probably Chinese or less likely Soviet, probably from a study on the effectiveness of H1N1 flu vaccines on young 18 year olds who were just coming through the demographic pipeline who had no immunity to H1N1 because it had been pushed out of the population by the 1957 H2N2 pandemic. It was a reassortment between a known human circulating strain that from 1950 and a second laboratory workhorse strain from the late 40s, paused for 27 years with zero evolution since their isolation, and re-emerged right when you would have human subjects available to test the effects of challenging adults who had no immune memory against H1N1 in an area that was heavily militarized and the earliest publicly known cases among military recruits.

(Amusingly enough this flu pandemic was the one and only reason that the 2009 swine flu pandemic was a nothingburger, the population had immune memory against a strain similar enough to prevent severe disease)

It just seems like calling the covid pandemic artificial became a community shibboleth along with suspicion about EVERY FREAKING DISEASE specifically and only because it gave them something to point to about the Dangers of Research!!!!!!!oneoneone

To complexity theorists, this is an extremely funny situation; I bet OpenAI started by asking the bot about P vs NP and they got the worst-possible result, a useless and unexpected refutation of a conjecture. To be clear, I don't trust Lean or OpenAI, so I don't yet accept this result, but I hear Ryan Williams and Russell Impagliazzo are hand-checking the proof and I do trust them.

Tesla Semi went up in flames on I-880 in the Bay Area: https://www.nbcbayarea.com/news/local/tesla-semi-fire-i880-hayward-traffic/4165797/

Between this and the carload of freshly manufactured Teslas that burned to the ground on the Grapevine on I-5 last year, that seems like only one (1) freeway-closing big-rig related explosion per year. That's not so bad, right?

The Internet's emotional support smut is offline:

We estimate that service restoration for #AO3 will take around 2 weeks but it could be up to 4 weeks. We are hopeful that we will be able to bring the site back up in a degraded state sooner. If that does happen, there may be times when the site has to be brought down temporarily until this is fully resolved. Please stay tuned for further info.

Wonder how many of their users have transitioned to beating off to artisanal probabilistic LLM smut.

I'm going to guess "few", on general vibes.

I really hope so, but tbh my immediate thought was that they’d all be on “r/myboyfriendisai”

The boosters hope that the AI will replace mathematicians, but they are running into an awfully inconvenient fact: theorems and proofs are useless without people who care enough to understand them and make meaning from them.