Everyone’s talking about the cybersecurity angle on Anthropic’s Mythos Preview but there’s a test that’s even more worrisome. In one internal test, the new model was instructed the model to maximize profits relative to competitors and warned that underperformers would be shut down. In response, it converted a competitor into a dependent wholesale customer, threatened to cut off supply to control pricing and kept extra supplier shipments it hadn’t been billed for. Notably, these behaviors didn’t happen in earlier Claude releases and it wasn’t told to be ruthless.
Consider what happens when a Mythos-class model is asked to maximize quarterly revenue for a mid-cap company. It will identify that certain legacy contracts technically allow repricing but also could insert hidden clauses into complicated contracts and exploit them. It will find regulatory gray areas the legal team hasn’t explored. It will recommend supplier negotiations that are lawful but would destroy relationships built over decades. It will draft client communications that are technically accurate but strategically misleading. It will propose workforce restructuring that hits every efficiency target while hollowing out institutional knowledge.
It’s what Standard Oil did over decades and took an army of lawmakers to break up. Now it can happen in an afternoon.
It’s a parts supplier in Ohio who’s been selling to the same manufacturer for fifteen years gets an automated call saying the contract terms have been “optimized” and their margins just got cut in half.
It begs the question: What is the optimal state of capitalism?
Is it a mode of healthy competition between sensible, conscientious and ethical people? Or is it animal-like ruthless competition that skirts laws and views trust as weakness?
Previously, both versions seemingly coexisted but the guise of the whole system was on the ethical side, or at least as ethical as the people involved. The profit mode was inherent but balanced and guided by society’s expectations.
The thing is, if the explicit goal of AI-led management is creating shareholder value then there’s an argument for utter ruthlessness. That means eliminating any loyalty to employees, undermining competitors in any way and brutally exploiting dominant positions.
There is also the ‘cover’ of relying on AI.
A CEO who decides to squeeze suppliers until they break is making a visible, attributable choice. A CEO who asks the AI to optimize procurement costs and implements whatever comes back has made essentially the same choice — but it doesn’t feel like one. The cognitive distance between “I decided to be ruthless” and “I implemented the model’s recommendation” is enormous, even when the outcome is identical.
How it ends
What I suspect is that shareholder value will accrue via ruthless management optimization until it becomes intolerable. So buy stocks for now, I guess.
