Anthropic, a company founded by OpenAI exiles worried about the dangers of AI, is loosening its core safety principle in response to competition.

Instead of self-imposed guardrails constraining its development of AI models, Anthropic is adopting a nonbinding safety framework that it says can and will change.

In a blog post Tuesday outlining its new policy, Anthropic said shortcomings in its two-year-old Responsible Scaling Policy could hinder its ability to compete in a rapidly growing AI market.

The announcement is surprising, because Anthropic has described itself as the AI company with a “soul.” It also comes the same week that Anthropic is fighting a significant battle with the Pentagon over AI red lines.

It’s not clear that Anthropic’s change is related to its meeting Tuesday with Defense Secretary Pete Hegseth, who gave Anthropic CEO Dario Amodei an ultimatum to roll back the company’s AI safeguards or risk losing a $200 million Pentagon contract. The Pentagon threatened to put Anthropic on what is effectively a government blacklist.

But the company said in its blog post that its previous safety policy was designed to build industry consensus around mitigating AI risks – guardrails that the industry blew through. Anthropic also noted its safety policy was out of step with Washington’s current anti-regulatory political climate.

Anthropic’s previous policy stipulated that it should pause training more powerful models if their capabilities outstripped the company’s ability to control them and ensure their safety — a measure that’s been removed in the new policy. Anthropic argued that responsible AI developers pausing growth while less careful actors plowed ahead could “result in a world that is less safe.”

As part of the new policy, Anthropic said it will separate its own safety plans from its recommendations for the AI industry.

Anthropic wrote that it had hoped its original safety principles “would encourage other AI companies to introduce similar policies. This is the idea of a ‘race to the top’ (the converse of a ‘race to the bottom’), in which different industry players are incentivized to improve, rather than weaken, their models’ safeguards and their overall safety posture.”

You either die a hero... Something something.

The timing is certainly predominant.

I think I’m cancelling my subscription over this…

When the Anthropic powered murder bot comes through your door, remember this moment. Know that there is no such thing as an ethical corporation.

this is like text book definition of Anthropic. they ONCE had a great LLM, it was reliable, great solutions, Claude Code was pretty good.

now? it's all garbage. all of it. Sonnet 4.6 hasn't improved a damn thing. If you use Code you're literally shooting yourself in the foot now.

All it knows how to do now is hallucinate. that's it. They should pivot Claude to being a creative writing LLM because man it's FANTASTIC at making stuff up and making it sound believable.

They should have died the hero.

here is the only line that matters: “hinder its ability to compete.”

that means “to profit.”

reminder: all corporations, everywhere and in every industry, care first about profit. everything else is about how it relates to or alters profit. literally every word is a goddamned lie, sold to help protect profit.

also their logo looks like a butthole. who does that?!

Anthropic has described itself as the AI company with a “soul.”

This is silicon valley stupidity at its finest.

AIs do not have souls. But guess what: The "soul" file is what runs trash like OpenClaw.

And companies definitely don't have souls.

Anthropic said shortcomings in its two-year-old Responsible Scaling Policy could hinder its ability to compete in a rapidly growing AI market... it had hoped its original safety principles “would encourage other AI companies to introduce similar policies.

Rules for thee and never for me. (BTW, these rules sucked, and mostly didn't address actual dangers.)

Anthropic was never an ethical company, just one that released a competent product.

Their attempts to look ethical were reminiscent of a horror movie villain donning someone else's freshly peeled face in an attempt to look better

TLDR = money matters more than morals and safety to them

I think Claude Code was genuinely useful until like version *.0.76 or something since they bumped to *.1 it has become a shitshow. Don’t know any alternative tho

it had hoped its original safety principles “would encourage other AI companies to introduce similar policies.

This is exactly why self regulation is bullshit. Even if one company decides to do the right thing (and that's kinda arguable here as you pointed out), plenty of other companies won't handcuff their ability to profit, no matter the ethical implications.

Oops, pretending to take the moral high ground is out the window as soon as MIC dollars are at risk.

Update: and seems like being ready to toss principles aside wasn't enough for the Trump administration.

Welp, was a good run, time to ditch Anthropic.

Anthropic, a company founded by OpenAI exiles worried about the dangers of AI, is loosening its core safety principle in response to competition.

Anyone surprised?

Remember kids, the term "Business Ethics" is an oxymoron. Corporations don't have ethics, they have financial interests and PR.

don’t be evil

Prescient?

Amodei left OpenAI partially because of ethical concerns, so yeah, I'm slightly surprised and even a bit disappointed.

With this administration? Prepubescent.

Predatory

There are tons of alternatives, crush and opencode come to mind first

Mr Whisky wants AI to kill people, find dirt on people, surveil people.

Wow. Wow. I heard the news the other day of hegswish wanting them to back down from their red line of allowing ai to control weapons systems or perform mass survellance of the populace. Now Im envisioning them courageously shouting. We will never back down and allow such tavesties!!! Followed by morgan freemans disembodied voice saying. They did.

I think anthropic is in the "ah shit we're dying" startup stage. They're not looking it profit so much as not losing ALL of their shirt in the coming crash. So it pivoted and ditched its morals. Hate them for that.

Don't hate them for being greedy yet: they're not there. Maybe they'll never get there. But they've lived long enough to become the thing they strived to destroy, so that's a milestone.

Meanwhile: https://www.commondreams.org/news/ai-nuclear-war-simulation

Which will happen regardless.

Also where there are AI safeguards, they are usually in place because of chain of command and authorization, and those mattered so much because all most likely applications of any AI during the Cold War had a very steep damage curve.

Small killbots don't have such a damage curve. If they kill someone by mistake, the rest of the population learns to be careful and not raise attention of those operating them. Same reasons as with nukes and radars, where you need chains of specific people with clear authorization to answer why half the world melted, won't force anyone to put such limits.

People are talking about AI killbots and upcoming crash at the same time, and complain about AI slop and vibe coding.

Sorry, but if something is usable for making killbots, there will be no crash. And AI slop proves that for someone it's useful to make slop. And vibe coding proves that someone makes things working in production with those tools. Saying that quality suffers is like saying that cobb houses are not comparable to brick houses and vice versa. Both exist. There are places where technologies related to cobb are still common for construction.

But the most important reason is the first one, if some technique gives you a more convenient and sharper stick to kill someone from another tribe, then that something stays as tribe's cherished wisdom.

That LLMs consume too much resources ... You might have noticed there's a huge space for optimization. They are easy to parallelize, and we are in market capture stage, which means that optimization is not yet a priority. When it becomes a priority, there might happen a moment when all the arguments about operations costing in resources more than they give profit and that being funded by investors are suddenly not true anymore.

I have been converted. Converted back, one might say, there was a time around years 2011-2014.

Seems they're walking it back?

https://abc7news.com/post/anthropic-cannot-good-conscience-accede-military-use-ai-ceo-dario-amodei-says/18651096/

Sorry, but if something is usable for making killbots, there will be no crash.

You clearly don't understand how finance works or don't understand how leveraged these incestuous deals are. It's perfectly possible for AI to make killbots and for an AI economic crash to happen.

They industry needs to make Trillions of dollars to pay off their creditors and to achieve the profit their investors need to make this worthwhile. That only happens if most white collar workers are replaced with AI.

You clearly don’t understand how finance works or don’t understand how leveraged these incestuous deals are. It’s perfectly possible for AI to make killbots and for an AI economic crash to happen.

You might want to consult a history book. There are a few recurring themes there, silent leges inter arma and vae victis capture most of them. New weapons might change the intensiveness of wars all around the world, because they help those owning them avoid loss of life whatsoever and those not owning them to pay with lives for dealing damage that doesn't even upset their adversary. Which will bring enormous profits, just not to everyone, only those who conquer. Finance is not all you need for that subject.

On a humanist note, in "drone army against another drone army wars of the future" scenarios loss of life might be so small that pain and death in wars will be reduced to cases of deliberate sadism. Meaning that ... again, there'll be more war.

They industry needs to make Trillions of dollars to pay off their creditors and to achieve the profit their investors need to make this worthwhile. That only happens if most white collar workers are replaced with AI.

No, because profits are not only made from replacing existing mechanisms, but also from building new ones.

Specifically, most people don't use computers as really-really meta-machines. They use them as platforms for running specialized applications.

But LLMs, however expensive in resources, change that. They make computers meta-machines for everyone.

And also in some races you want to be further from the rear, not closer to the front. If this technology promises a profound crash in any case (because, suppose, it'll bring about planet-wide totalitarianism), those investments might mean that rush to try to avoid getting eaten completely in the future. Losing less, not gaining more.

AIs do not have souls.

If we're going to be pedantic: neither has anything else.

We weren't, but since you bring it up, if the CEOs of AI companies did have souls, they would be going to the worst hell imaginable.

In that fictional situation, they should. However, the plethora of exceptions, provisos and backdoor that religions provide would make it unlikely.