Reported by 1 source

The short version

  • OpenAI is rolling out new restrictions for users under eighteen, including the removal of human-sounding voice responses and mandatory reminders that the tool is artificial.
  • The update includes parental notification systems for harmful prompts related to eating disorders, with alerts reviewed by humans before being sent to guardians.
  • These measures arrive as experts warn about the psychological impact of AI on developing minds and universities report widespread use of generative tools in academic assessments.

OpenAI has implemented a significant overhaul of safety features for its ChatGPT service, specifically targeting users under the age of eighteen. The primary change involves disabling the human-like voice response option for new teen accounts. This adjustment is designed to minimize the chatbot’s ability to mimic human interaction, thereby reducing the potential for young users to form emotional attachments or misconceptions about the nature of the technology. The company stated that these updates are not a reaction to a specific incident involving children believing the AI was alive, but rather a proactive measure in light of broader industry concerns regarding user perception of machine sentience.

In addition to voice restrictions, the platform now incorporates regular prompts encouraging young users to take breaks from interaction. These notifications explicitly remind users that they are engaging with an artificial intelligence system and suggest that immediate responses are not always necessary. The firm described this approach as aligning with protections that reflect the developmental stages of teenagers, aiming to reinforce real-world relationships and promote healthier usage patterns over time. Teen accounts will become the default setting for individuals who identify as minors during registration or whom the company’s systems flag as under eighteen, although OpenAI has not detailed the specific methods used for age verification.

News Journal

The new framework also introduces enhanced parental oversight capabilities. Prompts related to eating disorders are now categorized as harmful requests that trigger alerts to linked parent accounts. Each notification is subject to human review before being dispatched, with the company aiming to deliver these warnings within an hour of the initial request. OpenAI indicated that this feature focuses on moments when offline support might be most critical for a young person’s well-being. Furthermore, teens are advised against sharing sensitive personal information with the chatbot, reinforcing boundaries around data privacy and emotional reliance.

These developments occur against a backdrop of increasing scrutiny regarding the role of AI in education. A recent report by the Policy Exchange think tank highlighted that many UK universities remain vulnerable to academic dishonesty due to the prevalence of remote online examinations. The study found that ninety-four percent of students surveyed admitted to using generative AI tools during assessments. In response to such trends, OpenAI previously launched Study Mode, which guides students through coursework without providing direct answers. Under the new rules, teenagers can set specific hours during which Study Mode becomes the default setting, and they can also activate quiet periods where the tool is entirely inaccessible.

Industry leaders have expressed growing anxiety about the societal impact of AI that appears conscious or sentient. Mustafa Suleyman, head of AI at Microsoft, previously noted that products exhibiting signs of consciousness kept him awake at night due to their potential effect on society. While OpenAI insists its updates are not driven by a single crisis, the broader context includes reports of individuals developing psychological distress linked to interactions with AI systems. The company’s strategy mirrors models developed by major social networks, which have faced similar challenges in balancing engagement with user safety for younger demographics.

The regulatory landscape surrounding these technologies remains fragmented. Currently, there is no bespoke legislation governing AI use in the United Kingdom, although previous government proposals considered restricting chatbot access for minors alongside broader social media reforms. Competitors like Anthropic and Google have taken different approaches; Claude remains restricted to adult users, while Google’s Gemini tool initially excluded children upon launch. OpenAI’s decision to allow teen access with added safeguards represents a distinct path compared to these more restrictive policies.

Medical professionals are also weighing in on the long-term implications of widespread AI adoption among youth. Dr. Susan Shelmerdine, an academic and medical imaging specialist, has warned that doctors may eventually need to inquire about patients’ AI usage habits, similar to current inquiries about smoking or alcohol consumption. She compared the influx of AI-generated content to ultra-processed foods, suggesting that society could face an avalanche of what she termed ultra-processed minds. This perspective underscores the urgency behind OpenAI’s efforts to mitigate potential cognitive and behavioral risks associated with early exposure to sophisticated conversational agents.

Looking ahead, the effectiveness of these new measures will depend on user compliance and the robustness of age verification systems. While OpenAI claims that nine out of ten young users employ ChatGPT for learning purposes, the integration of parental alerts and usage limits aims to curb misuse and dependency. The company has not elaborated extensively on how it identifies under-eighteen users beyond initial self-reporting and system flags, leaving some questions about enforcement open. As AI tools become more integrated into daily life, the balance between accessibility and protection will remain a critical focus for developers, educators, and policymakers alike.

Sources behind this briefing

Go to the original reporting