AI-Generated Content: The New Frontier

The Internet Watch Foundation's 2026 report documents an alarming escalation: 8,029 AI-generated CSAM images and videos were assessed in 2025, with AI-generated CSAM videos surging from 13 in 2024 to 3,443. Among AI-generated videos, 65% depicted Category A content (the most extreme), and 97% depicted girls (IWF).

NCMEC's 2025 data revealed 21.3 million total CyberTipline reports, with 1.5 million indicating a generative AI nexus, over 7,000 reports of users generating or possessing AI-generated CSAM, and 145,000+ reports of users employing AI to alter CSAM (NCMEC). The real-world impact on minors is tangible: 1 in 10 minors know someone who has used AI tools to generate nude images of other children (Thorn).

A critical nuance emerged in January 2026 when Stanford's Center for Internet and Society analyzed the NCMEC reporting data. The frequently cited figure of 485,000 “AI-related” NCMEC reports from the first half of 2025 was found to be misleading: 380,000 of those reports originated from Amazon, and none of Amazon's reports involved AI-generated CSAM. Instead, they were hash hits to known CSAM found in AI training data.

Stanford concluded that “nearly 80% of all 'Generative AI' CyberTipline reports involved no AI-generated CSAM at all” (Stanford CIS). This finding underscores the importance of data integrity in shaping policy responses — inflated statistics risk misallocating resources and distorting public understanding of the actual threat landscape.

Traditional hash-matching systems like Microsoft's PhotoDNA — which compares file fingerprints against verified CSAM databases with a false positive rate of approximately 1 in 50 billion — are fundamentally unable to detect AI-generated CSAM since it constitutes novel material with no existing hash signature (Microsoft). This gap has driven investment in AI-based detection: the DHS Cyber Crimes Center awarded a $150,000 contract to Hive AI specifically for AI-generated CSAM detection (MIT Technology Review).

Thorn's Safer platform represents the most scaled detection effort. In 2025, Safer processed 415.4 billion files, detected approximately 1.5 million known CSAM files through hash matching, and used AI to flag 3.84 million potential novel CSAM files for human review. The platform serves over 80 platforms and maintains a hash library of 6.3 million image hashes and 64 million video hashes (Thorn).

Apple's abandoned NeuralHash system illustrates the detection dilemma. Announced in August 2021, paused a month later, and formally abandoned in December 2022, NeuralHash would have performed client-side CSAM scanning of iCloud Photos. Apple concluded it “could not implement without ultimately jeopardizing the security and privacy of our users.” A class-action lawsuit filed in December 2024 alleges that Apple's abandonment facilitates CSAM proliferation on iCloud (CNET). Meanwhile, the EU backed down on mandatory CSAM detection orders in November 2025, opting for mitigation measures instead (9to5Mac).

The legislative response to AI-generated harmful content has been swift. The TAKE IT DOWN Act, signed May 19, 2025, criminalizes non-consensual intimate imagery including AI deepfakes, with penalties up to three years imprisonment and a 48-hour takedown requirement for platforms. The ENFORCE Act passed the Senate unanimously on December 16, 2025, equalizing criminal penalties for AI-generated CSAM with traditional CSAM (Thorn; Senator Cornyn).

Internationally, the UK's Crime and Policing Bill (February 2025) creates a new criminal offense for making, adapting, possessing, or supplying a “CSA image-generator.” First Amendment challenges are already emerging. In U.S. v. Anderegg, a court dismissed possession charges for wholly AI-generated CSAM citing Stanley v. Georgia, while allowing production and distribution charges to proceed — the first federal case heading to appellate court on AI CSAM and the First Amendment (Tech Policy Press).