What Next for Grok and xAI as Deepfake Crackdown Looms?

A wave of pressure is building on Elon Musk’s social media platform X and its AI chatbot Grok, with the UK Government set to introduce a law making it illegal to create non-consensual intimate images using artificial intelligence.

The move follows growing concerns around Grok’s role in producing fake, sexualised images of women and children, prompting coordinated action from global regulators.

Musk has responded by accusing the UK of authoritarianism. When one user claimed the UK arrests more people over social media posts than any other country, he replied: "Real fascism is arresting thousands of people for social media posts.”

AI-generated deepfakes to be criminalised

The UK's Technology Secretary Liz Kendall has confirmed that creating or requesting fake intimate images through AI will be made a criminal offence in the coming days. 

This change activates part of the Data (Use and Access) Act, which was passed in June 2025 but not enforced until now.

It comes as UK media regulator Ofcom launches a formal investigation into X over reports Grok has been used to create manipulated images of individuals in sexualised positions – often without consent.

Kendall said the new law would also criminalise companies that supply applications and software tools designed to create non-consensual content. This change forms part of the Crime and Policing Bill, currently going through UK Parliament. She describes the legislation as "targeting the problem at its source", adding that it does not restrict freedom of speech and is instead about "tackling violence against women and girls".

She added: "This is about upholding basic British values of decency and respect and ensuring the standards we expect offline are upheld online."

The government also plans to make the offence a "priority offence" under the Online Safety Act, which means both individuals and platforms could face prosecution.

“Let me be crystal clear," continued Kendall. "Under the Online Safety Act, sharing intimate images of people without their consent, or threatening to share them, including pictures of people in their underwear, is a criminal offence for individuals and for platforms."

On Monday, Prime Minister Sir Keir Starmer told MPs: "If X cannot control Grok, we will. The content which has circulated on X is vile. It's not just an affront to decent society; it is illegal.”

Global scrutiny mounts as platforms face legal risks

In a statement, Ofcom said it was examining whether X had failed in its legal duties, highlighting "deeply concerning reports of the Grok AI chatbot account on X being used to create and share undressed images of people – which may amount to intimate image abuse or pornography – and sexualised images of children that may amount to child sexual abuse material". 

If X is found to have broken the law, Ofcom can issue fines of up to 10% of the company’s global revenue or £18m (US$24.26m) – whichever is greater. Should X fail to comply, UK courts may be asked to force internet service providers to block access to the platform altogether.

Kendall urged Ofcom to conclude its investigation quickly: “It is vital that Ofcom complete this investigation swiftly because the public – and most importantly the victims – will not accept any delay."

Downing Street has also made its position clear, suggesting the government is open to stopping its use of X if the platform fails to act.

Deputy Prime Minister David Lammy said he had raised the issue directly with US Vice President JD Vance, who he says is sympathetic to the UK’s efforts.

Lammy told The Guardian: “This new technology is allowing deepfakes and the manipulation of images of women and children, which is just absolutely abhorrent.”

International action grows

Beyond the UK, multiple countries are now moving against xAI and Grok.

Malaysia and Indonesia have both restricted access to the chatbot over concerns about its ability to create fake, sexualised images. Malaysia’s Communications and Multimedia Commission (MCMC) said Grok can generate “obscene, sexually explicit, indecent, grossly offensive, and non-consensual manipulated images”, including those involving minors.

Despite assurances from xAI that the image-generation tools are limited to paying subscribers – who can be identified – Malaysian regulators say this fails to address the risks. They criticise X’s reliance on user-initiated reporting and demand stronger technical and moderation safeguards.

In Indonesia, Communications Minister Meutya Hafid says non-consensual deepfakes represent a “serious violation of human rights, dignity, and the security of citizens in the digital space”.

Elsewhere, Australia’s Prime Minister Anthony Albanese has condemned the use of generative AI to exploit people, calling it “abhorrent”.

In Germany, Culture and Media Minister Wolfram Weimer urged the European Commission to act, warning of the “industrialisation of sexual harassment”. European neighbours France and Italy have also responded, with the former's media regulator Arcom alerted and the latter's data protection authority warning that such tools can breach privacy laws and possibly constitute criminal acts.

In India, the IT and electronics ministry has sent a formal notice to X over Grok-generated explicit images, giving the platform 72 hours to take them down and report on its response.

As scrutiny grows, the UK is set to become the first Western country to criminalise both the creation and the supply of AI tools used for non-consensual intimate image abuse – setting a precedent others may follow.