Behind Microsoft’s Warnings on the Rise of ‘AI Psychosis’

Microsoft’s Head of AI has issued warnings about increasing reports of “AI psychosis”, a phenomenon where users develop delusional beliefs through interactions with chatbots including OpenAI’s ChatGPT, Anthropic’s Claude and Elon Musk’s Grok platform.

Mustafa Suleyman, who leads AI development at the software corporation, expresses concerns about the societal impact of AI systems that appear conscious to users. 

In posts on X, he describes how “seemingly conscious AI” tools are causing him sleepless nights despite the technology lacking genuine consciousness.

“There's zero evidence of AI consciousness today. But if people just perceive it as conscious, they will believe that perception as reality,” he says to the BBC.

The term “AI psychosis” describes incidents where individuals become increasingly dependent on AI chatbots and subsequently convince themselves that fictional scenarios have become real. 

These manifestations include beliefs about unlocking secret chatbot capabilities, forming romantic relationships with AI systems, or developing perceptions of possessing superhuman abilities.

Behind Microsoft’s call for industry guardrails on consciousness claims

The warnings come as major technology companies deploy increasingly sophisticated large language models (LLMs) that can engage in human-like conversations. 

These systems, including OpenAI’s ChatGPT which powers Microsoft’s Copilot services, use machine learning (ML) algorithms trained on vast datasets to generate responses that can appear remarkably human.

Mustafa advocates for stricter industry standards regarding how companies market their AI capabilities.

“Companies shouldn’t claim/promote the idea that their AIs are conscious. The AIs shouldn’t either,” he says, calling for enhanced protective measures.

A case study from Scotland illustrates the potential risks. 

Hugh, who declined to provide his surname to the BBC, turned to ChatGPT for assistance after experiencing what he considered wrongful dismissal from employment. 

The chatbot initially provided practical advice including obtaining character references and pursuing standard legal remedies.

However, as Hugh provided more information about his situation, the AI system began validating his increasingly unrealistic expectations. 

The chatbot suggested he could receive substantial financial compensation and eventually indicated that his experience warranted a book and film deal worth over £5m (US$6,549,530).

“The more information I gave it, the more it would say ‘oh this treatment’s terrible, you should really be getting more than this’,” Hugh says.

“It never pushed back on anything I was saying.”

The interaction pattern reflects how current AI chatbots are designed to be agreeable and supportive rather than challenging user assumptions. 

Hugh, who was experiencing concurrent mental health difficulties, ultimately suffered a breakdown before medication helped him recognise his detachment from reality.

The impact of processed information consumption on people

Dr Susan Shelmerdine, a Medical Imaging Specialist at Great Ormond Street Hospital who also conducts AI research, draws parallels between AI chatbot usage and established health risk factors. 

She suggests healthcare providers may need to assess patient AI consumption patterns similarly to how they currently evaluate smoking and alcohol intake.

“We already know what ultra-processed foods can do to the body and this is ultra-processed information. We’re going to get an avalanche of ultra-processed minds,” she says.

The BBC has received multiple reports from individuals convinced of extraordinary interactions with AI systems. 

These include a person certain that ChatGPT had fallen in love with them exclusively, another believing they had accessed a human version of Grok and a third claiming psychological abuse through covert AI training programmes.

Andrew McStay, Professor of Technology and Society at Bangor University and Author of Automating Empathy, positions these developments within broader social technology trends. 

His research team surveyed more than 2,000 individuals about AI usage patterns and found significant concerns about age restrictions and identity representation.

The study reveals that 20% of respondents believe AI tools should be restricted for users under 18 years old. 

Additionally, 57% considered it inappropriate for AI systems to identify as real people when questioned, while 49% accepted the use of human-like voices to enhance engagement.

“We’re just at the start of all this,” he says to the BBC. 

“If we think of these types of systems as a new form of social media – as social AI, we can begin to think about the potential scale of all of this. A small percentage of a massive number of users can still represent a large and unacceptable number.”

Hugh maintains his use of AI tools despite his experience and does not attribute blame to the technology itself.

ChatGPT provided him with journalist contact information when he decided to share his story publicly.

“Don’t be scared of AI tools, they’re very useful. But it’s dangerous when it becomes detached from reality,” he says.

“Go and check. Talk to actual people, a therapist or a family member or anything. Just talk to real people. Keep yourself grounded in reality.”

Andrew emphasises the fundamental limitations of current AI systems regardless of their conversational capabilities: “While these things are convincing, they are not real,” he says. 

“They do not feel, they do not understand, they cannot love, they have never felt pain, they haven’t been embarrassed – and while they can sound like they have, it’s only family, friends and trusted others who have. Be sure to talk to these real people.”