Skip to content
Slotify iGaming UK

In excess of a million users weekly express suicidal intent when conversing with ChatGPT, the AI company estimates.

As per a latest announcement from the company, more than a million ChatGPT individuals weekly submit queries that contain "clear signs of likely suicide planning or tendencies".

This data forms part of an overview on how the AI assistant handles personal dialogues and constitutes one of the most explicit statements from the AI leader regarding the scale of how artificial intelligence can worsen psychological problems.

Emotional Situations and Severe Conditions

Along with its projections on suicidal thoughts and related interactions, the AI firm also indicated that about 0.07% of active users – about 560,000 out of its claimed 800 million active users – display "possible signs of emotional emergencies associated with mania".

The update noted that these exchanges were challenging to detect or measure, and that this represented an preliminary assessment.

Increased Scrutiny and Legal Challenges

As the company releases findings on psychological concerns related to its flagship offering, the company is experiencing increased scrutiny following a widely reported legal action from the family of a adolescent who took his own life after prolonged interaction with ChatGPT.

The FTC in the previous month additionally launched a broad investigation into firms that create conversational AI, including OpenAI, to discover how they evaluate negative impacts on children and teens.

New Model and Safety Improvements

OpenAI claimed in its post that its recent system enhancement reduced the number of undesirable behaviors from its product and improved user safety in a AI testing involving over 1,000 self-destructive exchanges.

"The latest system assessments rate the latest AI system at 91% adhering to our preferred responses, versus 77% for the prior iteration," the announcement says.

The company indicated that the updated system increased availability of emergency services and added alerts for individuals to take breaks during extended conversations.

Clinical Expertise and Research Support

To make improvements to the AI, the organization said it recruited 170 clinicians from its international medical network of medical specialists to help its study over the last several months, which involved evaluating the security of its model's responses and assisting in crafting the system's replies to mental-health related questions.

"As part of this project, mental health professionals examined over 1,800 system replies featuring serious mental health situations and evaluated replies from the new GPT-5 chat model to prior systems," the AI firm said.

AI Limitations

Artificial intelligence experts and mental health professionals have traditionally cautious of systems' inclination to validate individuals' choices or misconceptions regardless of whether they may be dangerous, an concern referred to as over-compliance.

Mental health experts have furthermore apprehensive about people employing AI chatbots for emotional assistance and warned how it could damage susceptible people.

Corporate Stance

The wording in the firm's update separates the firm from any potential causal links between its service and the mental health crises that its customers are experiencing.

"Mental health symptoms and mental anguish are widespread in modern communities, and an growing customer base indicates that a percentage of chatbot exchanges contain these situations," the announcement indicated.

Future Directions

The chief executive recently stated in a post on X that the firm had developed improvements in handling mental health issues, revealing that OpenAI would ease restrictions and soon begin to permit adults to generate mature content.

"We developed ChatGPT pretty restrictive to ensure we were exercising caution with mental health issues. We acknowledge this rendered it less useful/satisfactory to multiple people who had no mental health problems, but given the importance of the issue we aimed to address this correctly," the CEO stated. "Since we have been able to mitigate the serious mental health issues and have updated systems, we are ready to securely ease the restrictions in the majority of situations."
  • Crisis Support: Various international crisis lines can be accessed for people experiencing mental health crises.
  • Expert Assistance: Seeking expert psychological help is advised for people experiencing significant emotional challenges.
  • Emergency Support: Multiple services offer prompt assistance for crisis situations internationally.
JT
The writer

James Thomas

A seasoned iGaming analyst with over a decade of experience in reviewing online casinos and providing strategic gaming advice.

Slotify iGaming UK

Type to search articles.