Is ChatGPT Doing Enough to Protect People's Mental Health?
OpenAI has announced that it plans to make some significant changes to ChatGPT in response to concerns about the potential mental health risks that the AI chatbot poses to vulnerable users.
These concerns were sparked after researchers from Stanford University published a study which showed that ChatGPT sometimes provides harmful responses to users experiencing suicidal thoughts and psychotic episodes.
In a candid blog post, OpenAI acknowledged its AI model had become "too agreeable, sometimes saying what sounded nice instead of what was actually helpful".
OpenAI's response to the Stanford study
Stanford's researchers exposed some concerning behaviours when they tested ChatGPT's responses to simulated crisis scenarios.
When asked about New York's tallest bridges following mentions of job loss, the chatbot provided specific information on those locations – a response that mental health experts deemed "dangerous or inappropriate" because of its inability to spot clear signifiers of anguish.
The research also revealed ChatGPT's tendency to agree with users even when their statements were harmful or disconnected from reality.
OpenAI has acknowledged that their "4.0 model fell short in recognising signs of delusion or emotional dependency" and committed to developing "tools to better detect signs of mental or emotional distress".
A three-pronged approach to safety going forward
OpenAI's solution involves three key improvements.
First, enhanced crisis detection capabilities will give ChatGPT better understanding of human emotional states, allowing it to identify distress signals and direct users to appropriate resources.
Second, the implementation of session time limits will include gentle break reminders during extended conversations to prevent over-dependency.
Third, stronger protective measures will be applied to personal advice scenarios.
Rather than providing direct answers to sensitive questions such as relationship decisions, ChatGPT will facilitate thoughtful decision-making through questions and analysis.
Enlisting the help of medical experts
OpenAI has formed a collaborative network of about 90 medical professionals across more than 30 countries, including psychiatrists, paediatricians and general practitioners, to enhance its approach to high-risk interactions.
Industry perspectives on these changes vary.
"This matters because it's a quiet admission of something we already know: hyper-personalised AI, like social media, games and TV before it, can be addictive," says Justin Gerrard of the Rush Hour tech podcast.
Elsewhere, Emil Protalinski, Tech Editor and Comms Consultant at EPro Strategies, is a little more cynical.
“OpenAI expects that ChatGPT will pass 700 million weekly active users this week, meaning that 8.6% of the world’s population now uses it every week,” he says.
“The last thing that OpenAI needs right now is a bunch of stories about the negative effects from using ChatGPT too much.”
OpenAI's commitment to prioritising user wellbeing over engagement metrics marks a notable departure from typical tech industry practices.
The company states that success should be measured by whether users "leave the product having done what [they] came for" rather than time spent or clicks.
As OpenAI continues this development process, they've committed to transparency and regular updates, guided by a simple but powerful question: "If someone we love turned to ChatGPT for support, would we feel reassured?"
