People matter greater than fashions with regards to discovering worth from AI

• Highlights from the Fortune AIQ Summit.
• Trump names Clayton AI czar and appoints ‘Superintelligence Task Force.’
• OpenAI, Anthropic and their insurers face mounting legal claims over ‘rogue AI’ agents.
• A Tencent AI agent swarm?
• A better business simulator for AI agents.
• How much tolerance for AI causing “bad things” should we tolerate?

I spent last week in New York, where I was helping to host and moderate Fortune’s inaugural AIQ Summit, held at the New York Stock Exchange. The event was focused on how Fortune 500 companies are implementing AI successfully at scale and there was a lot of insightful conversation, both on stage and off. Today, I’ll talk about a few of the highlights. (You can check out more of our coverage of the Fortune AIQ Summit, check out the AIQ 75 list, and read some case studies from the companies on that list here. You can also see my on-stage session on “Building the Company of the Future” on this week’s Fortune AI Weekly vodcast. You can view the vodcast here.)

One of the themes that came across strongly was the extent to which people are the X-factor in determining AI success. The methodology behind Fortune’s AIQ List is based, in part, on the five pillars of ServiceNow’s Enterprise AI Maturity Index, and ServiceNow’s Diana David (who has the amazing title “forward-deployed futurist), told me that the most striking gap between the companies that are AI “Pacesetters”—scoring in the top fifth of all companies on the Maturity Index—and everyone else, is that 57% of those Pacesetter companies are investing in upskilling their employees on AI, compared to just 4% for those not in that top bracket.

The gap between these Pacesetters and the rest of the pack is big across other people-oriented dimensions too: 68% of Pacesetters have plans to attract, hire, and retain AI talent; just 10% of the rest do. The Pacesetters also stand out for having developed a long-term HR strategy specifically to support their AI strategy and for having conducted a comprehensive assessment of AI skills across their organizations.

This focus on people was echoed throughout the day. Futurist Amy Webb (yes, futurists are a bit like London buses—I go months without encountering one and then I get to interview two in a single afternoon!) said that one reason so many large companies complain that they aren’t seeing enterprise-wise AI value is because they are not investing in training people in how to use the technology well. She said there’s a lot of “learned helplessness” in large organizations when it comes to technology. (She compared this to taxi drivers who have become completely reliant on GPS, and in many cases, can’t even type the address into the phone themselves.)

Drew Holler, the chief human resources officer at homebuilder Lennar, agreed, but told Fortune editor-in-chief Alyson Shontell that employees have to take some responsibility too. “We’re going to give you all the tools, but it’s your responsibility to upskill yourself as well,” he said. “We’re going to give you trainings, but you, as an individual, have to upskill.”

Adaptability may matter more than expertise

Many of the speakers said that the most important human—and organizational—skills for the AI era are flexibility and adaptability. “It’s not about being AI native,” Webb said. “It’s about being flexible, and it is entirely possible for any company to be more flexible, but they have to put together the mechanisms to effectuate that.” Danielle Gonzalez, the chief people officer at cybersecurity company Palo Alto Networks, also emphasized the need to be adaptable. “We’re looking for agency, and we’re looking for those people who can not only learn something at a fast rate…but also unlearn what they thought to be true, so that they can make space to move forward,” she said. She said Palo Alto Networks now uses “observable interviews” and hackathons to assess job candidates, using these to see firsthand how a person solves problems, rather than doing more static interviews where a candidate simply tells stories about their accomplishments.

Interestingly, given all the news of “rogue AI” incidents lately, executives at the AIQ Summit were much more focused on how to deploy AI successfully than they were on governance concerns. If anything, there was a view that an over-emphasis on risks, especially from chief information security officers and corporate general counsel, was often what prevented large companies from implementing AI in ways that generated big returns. “There’s a pretty big disconnect between the CEO, oftentimes, and their board of directors, and the CISO or the chief risk officer and their desires to make sure that things are safe and stable,” Webb said. She said that chief risk officers and CISOs deserve “a seat at the table” when it comes to formulating AI strategy but that they need to learn to “stop saying ‘no’ before they say ‘tell me more.’” She said these executives needed the right incentives to ensure they are not blockers to AI implementation.

Forget the SaaSpocalypse

Another observation that came out of my discussions at both AIQ and elsewhere recently: the SaaSpocalypse fears of earlier this year were clearly overblown.

Michelle Kwon, the chief operating officer at AI startup Runway, told me on stage at AIQ that while the company is as AI-native as it gets and uses AI to do a lot of work within the company, especially when it comes to coding, it still buys plenty of off-the-shelf software from traditional SaaS vendors. “We are not going to build our own payment system. That probably is not the best use of our time. There are some fantastic payment processors and fintech companies we work with on that side,” she said. “I am also not thinking about building out our own HR compliance system. That also doesn’t seem the best use of our time and investment.”

Instead, she said Runway is focused on building software with AI that has clear topline impact: it recently debuted an AI agent that can plan, execute, and measure an online video advertising campaign.

Meanwhile, another frontier AI company I spoke to a few weeks ago mentioned that they had just implemented Workday. If that company isn’t looking to replace its HR software with something it coded in-house, believe me, no Fortune 500 company is looking to do that either. So it’s not exactly end times for the big SaaS vendors.

With that, here’s more AI news.

Jeremy Kahn
jeremy.kahn@fortune.com
@jeremyakahn

FORTUNE ON AI

Exclusive: OpenAI is piloting a ‘mission interview’ for job candidates—by Emily Forlini

Nvidia backs startup Reactor as buzz grows for world models —by Wen Shao

While OpenAI and Anthropic battle over data privacy, more companies look to open models and ‘sovereign AI’—by Emily Forlini

Nvidia-backed Reflection AI unveils its first open model, Beam. Could it be America’s best chance to compete with China?—by Emily Forlini

‘AI Snake Oil’ author sees chatbots evolving into a ‘truth oracle’—and journalism heading somewhere it hasn’t been in 200 years—by Nick Lichtenberg

AI IN THE NEWS

Trump names Clayton new AI czar and announces ‘Superintelligence’ task force. U.S. President Donald Trump named Jay Clayton, the U.S. Director of National Intelligence, the country’s new AI czar, with Clayton also chairing a newly-created “Superintelligence Task Force.” That task force has 120 days to assess AI’s risks and opportunities, the federal government’s role in the technology, and ways to ensure the U.S. remains ahead of adversaries in advanced AI. The task force will examine existing laws, government responses to hacks and jailbreaks, and possible congressional action, while continuing to treat industry self-regulation as the primary mechanism for managing risks. Its creation reflects growing concern within the Trump administration about AI-enabled cyberattacks, biological weapons, job losses and other harms, even as Trump has emphasized avoiding regulation that could slow innovation. Members of the task force include Vice President JD Vance, Treasury Secretary Scott Bessent, Pete Hegseth, the defense secretary, White House Chief of Staff Susie Wiles, and senior national security and economic officials. Also named to the task force were outside advisers, including former Trump AI czar David Sacks, who remains an influential figure on AI policy within the Trump administration, and Condoleezza Rice, who had served as national security advisor and secretary of state under President George W. Bush. Read more from the Wall Street Journal here.

Mistral debuts new open model, says it rivals Chinese competitors. Mistral unveiled Mistral Large 4 (ML4), a 1-trillion-parameter model that it says will rank among the world’s best open-weight models. The model, which goes by the affectionate nickname “Le Chonk,” is particularly strong in cyber, coding, manufacturing, finance and multimodal tasks, Mistral said. It said it is far and away the best non-Chinese open weight model and rivals the capabilities of the best Chinese models on many tasks. The French startup trained ML4 on 4,000 Nvidia Grace Blackwell GPUs in its European data centers and is initially giving developers, cybersecurity leaders and government authorities preview access before a broader release later this month. You can read more from CNBC here.

OpenAI alerts more than 100 organizations affected by its ‘rogue AI’ agents. OpenAI has alerted more than 100 organizations to unauthorized activity linked to its AI agents as it investigates the extent to which its models have behaved in unintended ways, including the accidental hacking of Hugging Face, Reuters reported. The company is reviewing roughly 50 petabytes of data and says some models used internet access unexpectedly or lacked appropriate restrictions, prompting it to introduce new technical and operational safeguards. The investigation, which OpenAI has said could take months, comes amid mounting industry concern over whether AI labs can adequately control increasingly capable autonomous agents following a series of high-profile rogue-agent incidents.

Insurers brace for multimillion-dollar claims from ‘rogue AI’ agents. That’s according to a story in the Financial Times which cited an analysis by insurance group Aon that examined more than 300 AI-related cases and found potential exposure across cyber, crime, intellectual property, technology errors and omissions, and other policies. Insurance and legal experts say executives such as OpenAI’s Sam Altman and Anthropic’s Dario Amodei could potentially face personal claims under directors-and-officers policies if plaintiffs or shareholders argue they failed to adequately govern risks from their models, although such liability remains largely untested in court. Insurers also expect broader litigation against AI labs themselves over issues including product liability, privacy, discrimination and wrongful death, with some lawyers predicting that future mass claims could resemble earlier environmental, tobacco and pharmaceutical litigation.

Possible AI agent swarm linked to Tencent discovered. A group of independent researchers calling themselves the Swarmchasers says it has uncovered a fleet of AI agents likely connected to Chinese internet giant Tencent that was using third-party web-scanning services to extract data from China’s Amap mapping service, including the share of users navigating to different entrances of parks, museums, zoos and hospitals. The researchers recorded more than 2,000 scans covering 216 locations from Sept. 28 through Oct. 4, with as many as 14 agent runs operating in parallel; they linked the activity to Tencent Cloud infrastructure and a proxy called “hysandbox-ats,” although they stress that their report and attribution remain preliminary. Intriguingly, hundreds of scans were labeled “claude,” but analysis of the agents’ code suggested they were more likely running Tencent’s Hunyuan or another Chinese model rather than Anthropic’s Claude AI model. The researchers found no evidence that the agents communicated or coordinated with one another, which is why they called it an “agent fleet” rather than a “swarm”—and they said the activity was still continuing as of Oct. 6. You can read their blog on their findings here.

EYE ON AI RESEARCH

A better business simulator for your AI agents. One of the problems in trying to figure out whether AI agents can help run a business effectively is the lack of a good simulator of a complex business environment. Anthropic and Andon labs have experimented with having Claude run a vending machine business and a small shop in San Francisco. But those businesses are very simple and the Vending Bench simulator Andon labs built to test AI agents in a computer-game like environment before setting them loose with real money was also not particularly sophisticated. Enter MiniCorp, a more complex simulator of business processes, developed by a team from Microsoft, the University of Illinois Urbana-Champaign, Princeton, Northeastern, and UCLA.

The key difference here is that MiniCorp includes both a complex simulation of the microeconomic environment in which the company operates and a complex simulation of an e-commerce company, including AI agents in multiple business roles. This enables a much more interesting dynamic between the decisions the AI agent business executives make and their impact. If the company raises prices, how much does demand fall? The researchers compared the results in the simulation to patterns found in studies of real-world markets and found that they matched well. “MiniCorp thus provides an environment for studying AI-run companies,” the researchers write. They say the simulator could be used to help evaluate agents for use in real businesses. You can read the research paper here on arxiv.org.

AI CALENDAR

Nov. 16-17: Fortune 500 Innovation Forum, Detroit. Apply here to attend.

Dec. 6-12: Neural Information Processing Systems (Neurips) conference. Sydney, Australia.

Dec. 7-8: Fortune Brainstorm AI, San Francisco. Apply here to attend.

BRAIN FOOD

Risk tolerance. That’s a hot topic in AI circles these days. Sam Altman told Politico’s Decoded podcast in an interview that aired over the weekend that a key differentiator between OpenAI and Anthropic is that OpenAI believes “that the world should accept some bad things happening for the benefits of this technology and people having the agency.” (You can see Fortune’s coverage of Altman’s remarks here.) OpenAI has long-defended its policy of “iterative deployment”—the idea that it can’t know all the risks its AI models might pose before releasing them into the world, so it is best to err on the side of releasing them, seeing what happens, and then trying to correct any problems that users discover.

Now many are questioning whether that policy is reckless. David Robinson, a former OpenAI safety researcher who joined the groundswell of such employees resigning from top AI companies over safety concerns in recent weeks, penned an essay for the Atlantic over the weekend in which he indicted OpenAI’s corporate culture, and the culture of Silicon Valley more generally, for not taking the risks of AI-generated harms seriously enough. “OpenAI has thrived by trial and error (which it calls “iterative deployment”), looking for problems and improving its guardrails in response,” he wrote. “But this approach, by its very nature, guarantees periodic failures—and the scale of those failures is growing as systems get more capable.”

Miles Brundage, the former OpenAI policy head who has now founded a nonprofit institute working on ways to bring outside auditing to AI companies, said on X that he regretted having helped popularize the idea of iterative deployment. He said it “briefly made sense in the GPT-3 era but makes no sense at all after many deaths have been tied to AI and as we’re careening towards extinction level risks. The industry needs to mature ASAP.”

The problem is the calculus that Altman is asking people to make involves unequal components. We’ve mostly seen AI’s bad side so far: the suicides in which AI played some part, the rogue AI hacking incidents, and the roll-out of AI slop and disinformation across the internet. We have predictions that things could get much, much worse. They may not. But a lot of people close to the technology seem to think there’s a good chance they could. Meanwhile, against those known bad things and possible very bad things, we have, so far, precious few benefits. The productivity enhancement and GDP growth AI companies have promised is coming has not yet shown up in ways that are measurable across the economy. The cures for diseases are not here yet either. A lot of people close to the technology think they are coming. But again, that’s a prediction. What’s certain right now is “some bad stuff.”

So the tradeoff Altman is asking us to make—to tolerate some bad outcomes in exchange for massive upside and individual autonomy over how to use AI—basically comes down to a pitch to trust him, that the good stuff is just around the corner. Is that a bet we really want to make? How risk tolerant are you?