Data privacy has emerged as a key consideration for companies when choosing an AI vendor, amid fears that frontier AI companies, such as OpenAI and Anthropic, may be quietly using their customers’ intellectual property to improve their models.
Some executives worry that using AI models from these companies will eventually result in their “secret sauce” leaking to competitors—or worse, that OpenAI and Anthropic will build products that directly compete with them.
The biggest controversy came in June, when Anthropic began mandating a 30-day retention policy for all chats with its most powerful AI models, Fable and Mythos. Many companies had an “almost allergic reaction” to this, says a source who advises clients on becoming AI-native, speaking under the condition of anonymity because his company works closely with Anthropic. “The industry at large totally rejected it, and was pretty angry about it.”
A month later, both Microsoft CEO Satya Nadella and Palantir CEO Alex Karp warned companies against relying on AI models from frontier AI companies, saying that they risked having the IP siphoned off. “You essentially pay for intelligence twice, once with money, and again with something even more valuable: the proprietary knowledge you must reveal to make that intelligence useful,” Nadella said. OpenAI and Anthropic have stated they do not train their models on enterprise data.
Still, many data security experts say the AI companies may have subtler ways to glean insights about how their customers’ operate that can be used to improve future AI models, without having to train directly on the data the models’ are processing. “From a business standpoint, the risk of any of that data being trained on an AI model is unacceptable,” Rob Gregory, Vice President and Chief Information Security Officer at Optiv, a B2B cybersecurity advisory company, tells Fortune. “We’re at a very interesting crossroad.”
The latest string of “rogue AI” incidents have heightened fears that the latest models will either scrape data, or publish it for the world to see, without the business’ permission.
Given these concerns, companies are looking to diversify their AI usage, and reduce their dependency on the leading AI companies. The term du jour for this is “sovereign AI,” referring to a company’s ability to control its own “AI stack”—everything from owning the chips on which AI workloads run, to setting up proprietary cloud infrastructure, to using open source models that a company can download and run in their own cloud environments.
Sovereign AI is a term that has historically been associated with governments, but this year its usage has bled into the corporate world.
“Data sovereignty is something we’re talking a lot about,” said Brooke Hopkins, founder of voice AI company Coval. Dutta also said sovereign AI has “been a large topic of conversation for us over the past few months.”
The problem is that running open source AI models requires more technical expertise and, in some cases, may also mean that companies are not getting the cutting-edge capabilities, in areas such as coding or financial analysis, that some of the products from the likes of OpenAI, Anthropic, and Google offer. “You’re trading control for responsibility, right?” Gregory said. “Because there is a significant amount of responsibility when you bring that internally.”
OpenAI and Anthropic are still earning trust
Anthropic and OpenAI are competing directly to win over business customers, and keep the ones they have from either switching companies or seeking open source alternatives.
OpenAI saw an opening to win customers after Anthropic’s data retention announcement, and in an August 19 blog post reiterated that it offers full ZDR for enterprise customers, while also previewing a new feature called Private Safety Processing (PSP). PSP adds a data storage element to ZDR, while also offering a system in which customers can store data in their own cloud, rather than on OpenAI’s servers.
Anthropic responded on Sept. 1 with a new, similar version of ZDR that also offers businesses a way to store their data on their own cloud accounts—not Anthropic’s. But has the damage already been done? After the company announced its 30-day retention policy, businesses like Booz Allen decided to restrict their teams’ use of its Fable model, The Information reported.
Another common route for businesses to use leading models from these companies is through Amazon Bedrock. Bedrock is an entrypoint for companies to access multiple AI models, without the providers seeing their data. It’s one of the fastest growing AWS products of all time, with customer spend growing 170% in Q1, and nearly 80% adoption by the Fortune 100.
“Amazon is like, ‘You don’t have to trust the frontier labs,’” said Randall Hunt, chief technology officer at Caylent, an AWS advisory company. “It’s a more established company people have been using for decades.”
Open models require more oversight—and obtaining GPUs
Many companies are adopting a “hybrid model,” Gregory said, meaning they are using some combination of closed models and open source models.
“Sovereign AI basically enables you to say, ‘Okay, I will continue to partner with OpenAI and Anthropic for some things, but maybe here’s another set of use cases that I want to use open source models for on prem,” Dutta added.
But this requires companies themselves to secure the models and implement them responsibly. Glen Wise, CEO and co-founder of Cinder, which makes software that helps detect and remediate model abuse, has seen a spike in inbound customer requests since Anthropic announced Fable would not have ZDR. “We’ve already been working with the main model creators, but in the past few weeks we have started getting requests from regular companies that are trying to figure out how to deploy open models,” Wise said.
Businesses with particularly sensitive data, such as health records, are not shying away from these added responsibilities, and some are even taking it a step further by running open models on their own GPUs, essentially becoming miniature data center owners. Payroll company Paycom told Fortune it has taken this approach because of the sensitive customer data and financial information it handles.
This is still relatively uncommon, but some people think it will prove to be the smartest long-term strategy.
“There’s a big shift going on, whereas in the past 25 years, [businesses would] pick your hyper-scalers, and then just let them take care of all things compute,” said Dan Wright, the founder and CEO of Armada, which makes portable data centers and will provide the infrastructure for Palantir and Nvidia’s forthcoming joint offering to help other businesses adopt fully sovereign AI. Crusoe,another company that has pivoted from building massive data centers to focusing on tiny ones. It was recently valued at $31 billion in a venture capital round earlier this month, according to the Wall Street Journal. “This is very much the direction the market is going,” Wright said.
But not every company is as anxious about data and IP leaking to the AI companies. “While I need to be paranoid about my IP, I trust the big AI companies are going to do the right thing because we’re both on the line to make sure it’s secure,” said Dan Cane, co-CEO of ModMed.
