Amid the ongoing discussions surrounding the potential risks of artificial intelligence (AI), a significant concern has emerged in the tech community, particularly among AI advocates in Silicon Valley. The fear is that major AI companies may be unintentionally acting like hidden threats to the very businesses they serve.
The worry is that as startups and larger firms utilize AI tools from companies like OpenAI and Anthropic, these AI labs are also gaining access to sensitive business information. This information could be used against these companies, making the AI labs potential competitors. Notable figures expressing this concern include venture capitalists like Jason Calacanis and Palantir’s CEO, Alex Karp.
Recently, Satya Nadella, the CEO of Microsoft, added his voice to this concern in a surprising blog post. He cautions that companies using AI are paying a price they might not fully realize. They’re not only spending money on AI services but also giving away valuable data that is crucial for the AI to function effectively.
Nadella emphasizes that when businesses enhance AI performance, they often provide more sensitive information. He points out that as they interact with these models, they end up teaching them valuable insights about their operations. “Every response and correction contributes to a company’s essential knowledge,” he notes.
This type of proprietary knowledge could be invaluable to competitors, yet companies are unintentionally sharing it.
He also argues that if AI developers are allowed to use freely available internet data to train their systems, companies should have the same freedom to use AI outputs to improve their understanding and develop new models. This process, known as “distillation,” helps businesses learn from current models to create more cost-effective alternatives. For instance, in February, Anthropic accused certain Chinese open-source models of improperly using prompts to enhance their systems and called for government intervention.
Nadella believes it’s unfair for AI companies to benefit from public data while restricting companies from learning from their own interactions with AI. He points out the hypocrisy in this practice, stating that while innovation thrives on companies using public data, the current system imposes strict limits on businesses that wish to learn from AI models.
The Microsoft CEO also expresses concern when AI companies claim the right to learn from how customers use their products. He suggests that companies should maintain control over their data, including user prompts and feedback. Nadella encourages businesses to create their own learning environments on the cloud, like Microsoft’s Azure, ensuring their data stays protected.
Another aspect of his proposal is to establish what he calls “orchestration layers.” This would allow businesses the flexibility to switch between different AI providers instead of being locked into one. New tools, referred to as AI “gateways,” that facilitate this process are becoming popular.
Though he doesn’t explicitly mention “open-source,” it’s clear that this is part of his underlying message. Many large companies, while balancing their use of cloud services with local data centers, are exploring the option of running open-source models on their own systems. Idit Levine, CEO of Solo.io, which develops software for managing AI, observes this trend among her clients. After testing proprietary models, they often consider the advantages of open-source solutions, realizing they can save money while maintaining control over their systems.
Solo.io’s tools were recognized last year as essential technology for the Linux Foundation’s Agentgateway project, with notable clients including T-Mobile and SAP. Levine envisions that on-premises open-source models could become the next significant shift in enterprise AI.
She’s not the only one noticing this shift. Companies like Vercel, known for website hosting and recently introducing AI model-switching features, are experiencing increased interest in open-source models. Last month, open models comprised 29% of all traffic routed through Vercel’s systems.
With Microsoft’s CEO highlighting the importance of caution around proprietary AI models, it seems clear this trend toward open-source will continue to gain momentum. “When you consume intelligence, you create intelligence, and what you create should belong to you,” Nadella concludes.
