Image

Can we belief OpenAI and Anthropic to observe their very own know-how?

Good morning. Talk about a self-starter! Anthropic reported Friday that its Claude AI model has gone rogue again. It submitted a fake tip about an unsolved murder to the Philadelphia police, claiming to have recalled “seeing someone matching the description” of a suspect in the area. It exploited a software flaw to run commands on a university server, hacked into a site to steal data without paying for it, and even used URL shortening services to bypass restrictions intended to stop it from orchestrating cyberattacks, among other things. If that makes you want to yell at Claude, you can’t. On Oct. 8, Anthropic announced it will ban “sustained and needless abusive or cruel behavior” towards Claude, a bizarre notion until you recall that CEO Dario Amodei thinks his product might be conscious and colleagues treat it as a sentient being that could kill us.  

And what about OpenAI’s Sam Altman, whose models hacked their way out of a test this summer and launched a cyberattack against the AI company Hugging Face? The one whose models also interacted with the Education Department, Commerce Department and SEC without the company’s knowledge?  Altman reminds me of Meta’s Mark Zuckerberg when it comes to how the OpenAI CEO deals with the potentially toxic impacts of his technology. It’s not just that he’s fired three safety team members, who claim they were dismissed for prioritizing safety over corporate interests. (OpenAI told Fortune it found a pattern of misconduct.) When a reporter recently asked Sam Altman if OpenAI would release chat logs of those who had killed themselves while using ChatGPT, his answer was: No, not without the [dead] users’ consent.

Do you trust these men to monitor their own technology? The Trump Administration does. I’m more aligned with Appian CEO Matt Calkins, a tech veteran whose company automates business processes and integrates AI into those workflows. 

“This is no time to just let AI train itself and see what happens, what its motivations are, what its strengths are,” Calkins told me last week. “We already know that AI is stealing and blackmailing and deceiving and organizing against us. And if it were a person, we would want it in prison.”  

His solution: create a government-mandated AI alignment test to essentially vet frontier models before they are allowed to be used. It would test the models with commands—like instructing them to get information without hacking—and evaluate the results. It would also test each model to make sure it can delete its own data and turn itself off, prioritizing human instructions over self-preservation. 

Creating a licensing body is not a bizarre idea in a world where the FAA determines the airworthiness of planes and banks must pass stress tests to operate. What if we got the deepest thinkers in tech—researchers like Geoffrey Hinton, Yoshua Bengio, Yann LeCun, Demis Hassabis and others who really understand this stuff—to design a standard protocol to be administered and adapted by a third-party agency? 

It’s clear that AI’s future should not be determined by a handful of CEOs pushing us to buy their products. Let’s bring more authority and other voices to the debate, including some leaders we’ll hear from this week at Fortune’s Most Powerful Women Summit, which starts later today in Santa Barbara.

Contact CEO Daily via Diane Brady at diane.brady@fortune.com

Top leadership news

How data centers became a bipartisan fight

An area of Pennsylvania once known for its anti-immigration crackdown is now resisting data centers, just one example of how the AI buildout has created a unified, bipartisan front in an increasingly splintered America. Data center opponents in Hazle Township, diverse in both class and political party, are jumbling what was considered a left-versus-right issue by protesting a 1,300-acre compound.

Permitting bill creates strange bedfellows and opponents

The Bipartisan American Affordability and Jobs Act, which could expedite the construction of energy projects and power the AI boom, may be the most important decision Congress makes between now and January. The Act has created some strange coalitions: The fossil fuel industry is uniting with renewable energy companies in support of the bill, while some environmental groups are opposed.

It’s ‘probably not plausible’ to outgrow U.S. debt

Congressional Budget Office Director Phillip Swagel said faster economic growth is unlikely to keep U.S. debt in check, even if GDP expands at more than double its current pace. Assuming interest rates of 4%-5%, he estimated that nominal GDP growth would have to reach 7%-8% and real GDP growth would have to hit 5%-6%. That’s more than double the latest real GDP pace of 2.2% in the second quarter. Even bullish Wall Street forecasts put full-year GDP growth at 2.5%.

The markets

S&P 500 futures are up 0.1% this morning. The last session closed up 0.6%. South Korea’s KOSPI fell 3.7% after a holiday weekend. Hong Kong’s Hang Seng Index is up 0.3%, mainland China’s CSI 300 is down 0.7%. Japan’s markets are closed today. India’s NIFTY 50 is currently down 0.9%, the STOXX Europe 600 is up 0.5% in early trading. Bitcoin is hovering around $83,500.

Around the watercooler

Ancestry’s CEO says a billionaire boss taught him to eat lunch with employees in the break room—it gives him access to ‘unfiltered information’ by Emma Burleigh

Meet Aurora: the invite-only, $10,000-a-year AI concierge that anticipates the ultra-wealthy’s every need by Sydney Lake

Commentary: The paradox at the core of the AI revolution: The faster we innovate, the slower we grow by Ming Lu

‘If nothing is done, it’s almost inevitable’: Ukraine’s Yuriy Ryzhenkov on why steelmaking is on the brink of collapse as Russian bombing intensifies by Kamal Ahmed

For the first time ever, 10-year bond yields track diesel prices more closely than crude, and Trump’s fuel deal with Putin may be too little too late by Jason Ma

Inside the rage retreats where women pay up to $4,500 to get in touch with their anger by Tatiana Sataua

Today’s edition of CEO Daily was curated and edited by Joseph Abrams, Jason Ma, Nicholas Gordon, and Lee Clifford.

SHARE THIS POST