Why Microsoft CEO is Slamming AI Labs Over Distillation

Microsoft’s Satya Nadella has publicly criticised unnamed AI labs across the tech sector for their stance on AI distillation, a process that involves training a smaller AI model with the outputs of a larger, more powerful one.

In a post on the social media platform X, Satya says that model makers complaining about the distillation process are being hypocritical over the AI training process. 

He adds that while ‘the great innovation that comes from model providers having fair use rights to train on public data is needed’, he finds it ‘ironic’ that the consensus among AI model makers is to impose ‘restrictive terms’ on distillation. 

This happens while they still reserve the right to learn from customer usage and interaction data themselves.

“If learning flows in only one direction, economic value converges toward the owners of the learning infrastructure rather than the creators of the knowledge itself,” notes Satya.

Anthropic attacks highlight growing sourcing row

Anthropic, OpenAI and Google DeepMind rely on work created by others to train their own AI models. ChatGPT, Claude and Gemini source their information from publicly available writing, images and forms of data. 

Many have voiced ethical and copyright-related concerns regarding this training system, which has led to legal action from major content creators. 

For example, BBC News reported that the BBC is suing Anthropic for non-consensual information sourcing.

While Satya did not name Anthropic outright in the post, evidence suggests it was strongly directed at the AI firm. His criticism of AI model makers complaining about distillation follows a post by Anthropic earlier this year on ‘detecting and preventing distillation attacks’. 

In a letter sent to South Carolina Senator Tim Scott and Massachusetts Senator Elizabeth Warren in June, Anthropic claimed that Alibaba had recently carried out the ‘largest known distillation attack’ against the AI firm’s data. 

Competitors can use distillation to acquire ‘powerful abilities from other labs in a fraction of the time and at a fraction of the cost, that it would take to develop them independently’, the statement said. Alibaba is yet to respond to the accusations from Anthropic.

Model outsourcing raises critical business risks

In the same post on X, Satya also warns that companies relying on major AI models are giving AI firms access to their own proprietary data. 

He adds that companies need to own their AI infrastructure and industry knowledge instead of relying on a single model vendor. 

Satya says they should also conduct their own evaluations and their own ‘learning loop’ to allow their AI technology to develop and improve over time.

Enterprises require strict security parameters to protect their proprietary developments. That is why enterprises need a real trust boundary for their human capital and token capital to compound, Satya says. 

He adds that the boundary must not be crossed by anyone or anything, ‘not even the intelligence exhaust, without consent’.

Similarly, Elon Musk has also publicly criticised the use of data sourcing by Anthropic to train its own AI models. 

In a February post on X, Elon accused Anthropic of ‘stealing training data at a massive scale’, saying it will have to pay ‘multi-billion dollar settlements’ for theft. 

“This is just a fact,” he adds.