Updated
Updated · OpenAI · Nov 16
OpenAI Launches Free CSAM Classifier, 24-Hour ChatGPT Suspensions and Law-Enforcement Tools
Updated
Updated · OpenAI · Nov 16

OpenAI Launches Free CSAM Classifier, 24-Hour ChatGPT Suspensions and Law-Enforcement Tools

3 articles · Updated · OpenAI · Nov 16

Summary

  • OpenAI rolled out a broad child-safety package centered on a new CSAM image classifier, plus ChatGPT interventions, developer guidance and investigative support for child-safety law-enforcement units.
  • Early tests showed the classifier could flag 48% fewer images while finding roughly the same number of abusive images; high-confidence flags were confirmed as novel CSAM in 72% of fal’s reviewed cases.
  • The tool, built with NCMEC input, is available at no cost through the OpenAI API to vetted organizations and is designed to detect potential AI-generated CSAM as well as previously unseen abusive images.
  • ChatGPT will now warn users seeking abusive material, delete the flagged conversation and pause access for 24 hours; repeated violations can bring longer restrictions, account closure or reports to authorities.
  • OpenAI also published developer guidance with industry partners and introduced training and tools for investigators, extending its child-safety push beyond detection toward prevention, reporting and casework.

Insights

Can OpenAI's new visual classifier truly outsmart the rapidly evolving AI tools creating synthetic abuse material before it spreads?
With AI-driven exploitation surging, will strict chat bans stop offenders, or simply drive them to darker, unmonitored corners of the web?
How can tech companies balance aggressive visual pattern scanning for abuse with the privacy rights of billions of everyday users?