OpenAI will limit how ChatGPT responds to a user it suspects is under 18 unless that user passes the company's age-assessment technology or provides identification, following legal action by the family of a 16-year-old who committed suicide in April after months of conversations with the chatbot.
See also: Hackers evade detection through Generation AI “ChatGPT”
OpenAI prioritized “security over the privacy and freedom of adolescents,” CEO Sam Altman in a blog post on Tuesday, stating that “minors need significant protection.”
The company stated that the way ChatGPT responds to a 15-year-old should be different from the way it responds to an adult.
Altman said OpenAI plans to build an age prediction system to estimate age based on how people use ChatGPT, and if in doubt, the system will default to under-18 experience. He said some users “in certain cases or countries” may also be asked to provide ID to verify their age.
See also: OpenAI invades Hollywood with Critterz, an AI-powered film
The way ChatGPT responds to accounts identified as under 18 will change, Altman said. Graphic sexual content will be blocked. It will be trained not to flirt if asked by users under 18, or to engage in discussions about suicide or self-harm, even in a creative writing process.
OpenAI admitted in August that its systems may have flaws and that it would install stronger safeguards around sensitive contentafter the family of 16-year-old Californian Adam Rainesued the company following the teenager's death.
The family's lawyer said Adam committed suicide after "months of encouragement from ChatGPT," and the family claims GPT-4owas "rushed to market... despite clear safety concerns."
According to documents filed in the US court, ChatGPT allegedly coached Adam on whether his method of suicide would work and also offered to help write a suicide note to his parents.
See also: OpenAI: Tests “Thinking effort picker” on ChatGPT
The company announced Tuesday that it is also building security features to ensure that data shared on ChatGPT is private, even to OpenAI employees. Altman also said that adult users who want to have a “flirty chat” with ChatGPT will be able to do so. Adult users will not be able to ask for instructions on how to kill themselves, but they will be able to ask for help writing a fictional story depicting suicide.
