Skip to content
The Digital Harm Project

For compliance, T&S operations, legal, and internal audit

The documentation, audit, and process side of CSAM compliance.

The companion to the strategic guide for executives — focused on the artifacts an external assessor or regulator will actually ask to see. Regulatory mapping across US, EU, UK, and Australia; the metrics that hold up at audit; vendor due-diligence; records retention; and a 90-day buildout sequence for teams starting late.

01 · Regulatory map

What you're complying with, by jurisdiction.

The CSAM compliance surface is unusually multilateral. A platform with global users is simultaneously subject to several non-aligned regimes. Map your exposure to the jurisdictions where you have users, employees, or counterparties — not just where you are incorporated.

  • United States. 18 U.S.C. § 2258A mandates reporting of apparent CSAM to NCMEC by US-based providers. The ENFORCE Act (December 2025) and TAKE IT DOWN Act (May 2025) extend criminal-equivalent treatment to AI-generated CSAM and impose a 48-hour platform takedown requirement for non-consensual intimate imagery. Section 230 does not immunize against federal child exploitation law.
  • European Union. The Digital Services Act (DSA) applies to all online services serving EU users — illegal-content takedown obligations, transparency reports, risk assessments for Very Large Online Platforms. The EU's proposed CSAM Regulation backed down on mandatory client-side scanning in November 2025 but retained mitigation requirements. Treat as live and evolving.
  • United Kingdom. The Online Safety Act 2023 requires user-to-user services to assess CSAM risk, take proactive measures, and cooperate with Ofcom. Enforcement opened in mid-2025 with fines up to £18M or 10% of global revenue. The Crime and Policing Bill (February 2025) creates a new offense for “CSA image-generator” tools.
  • Australia. The eSafety Commissioner issues legally enforceable transparency notices under the Online Safety Act — already invoked against Apple, Google, Meta, Microsoft, Discord, WhatsApp, Snap, and Skype. Industry codes for CSAM detection took effect for messaging and file-sharing services in 2024.
  • Canada, Germany, France, Singapore and others each maintain their own reporting and takedown frameworks. The pattern is convergent toward criminal-equivalent treatment of AI-generated CSAM, mandatory reporting, and short takedown SLAs. Build for the strictest regime you face; relax only where you can document why.

For deep context on the legislative trajectory, see Chapter 07: Prevention Strategies.

02 · Audit checklist

What an external assessor will ask to see.

Whether the audit is internal, customer-driven (enterprise contracts increasingly require T&S maturity attestation), or regulator-led, the artifacts you produce should already exist. The minimum production-readiness checklist:

  • Detection coverage matrix. Per content type (image, video, text, audio, AI-generated) and per pipeline stage (upload, storage-at-rest, sharing, export). Coverage should map to specific detection technologies — PhotoDNA, PDQ, TMK+PDQF, Safer, Hive, Project Arachnid Shield — with documented gaps where they exist.
  • Hash list provenance and refresh cadence. Document which hash lists you query against (NCMEC, IWF, Project Arachnid, SaferList), at what frequency, and the last successful sync timestamp. Auditors will ask for the sync log.
  • CyberTipline reporting evidence. Volume per period, time-to-report distribution, the named primary and backup reporters, the escalation path, and a sample (redacted) report walkthrough. The 18 U.S.C. § 2258A reporting requirement is the minimum demonstrable bar in the US.
  • Takedown SLA evidence. Time-to-takedown distribution for confirmed CSAM, for non-consensual intimate imagery (TAKE IT DOWN Act 48-hour requirement), and for general adult content. Document misses and remediation.
  • Moderator wellbeing program. Clinical-grade support contracts, exposure limits, rotation policy, screening-on-hire, and trauma-symptom tracking. This is now a contractual and reputational checkpoint, not optional culture work.
  • Records retention. Preservation schedule for evidence (matching law-enforcement preservation requests), report copies, training logs, vendor agreements, and DPIA / PIA outcomes for any client-side or server-side scanning.
  • Incident-response runbook. Triggers, ownership, communication plan (internal, external, regulator), notification thresholds, and post-incident review. The runbook should be tested in tabletop exercises at least annually.
  • Training records. Annual completion certificates for moderators, engineers, legal, and executive teams. Include role-specific tracks; generic anti-harassment training does not satisfy this.

03 · Metrics that hold up

The numbers to track internally and report externally.

Vanity metrics (“total content moderated”) are weak. The metrics that stand up at audit and in transparency reports are operational, ratio-based, and contextualized.

  • Detection-pipeline volume by class. Hash matches vs AI-classifier matches vs human review, broken down by content type. Reveals where your stack is weakest.
  • Time-to-detection (TTD). Wall-clock between upload and detection. For confirmed CSAM, sub-hour TTDs are the industry expectation. Track the p50 and p99.
  • Time-to-takedown (TTT). Between detection and content removal. Hard regulatory limits exist (TAKE IT DOWN Act, DSA, Online Safety Act); report both the SLA and your actual distribution.
  • Time-to-report (TTR). Between detection and CyberTipline submission. Expect this to be tighter than TTT.
  • False-positive rate. Reviewed and confirmed false positives over total flags, per detection technology. Helps you tune classifier sensitivity and documents reasonableness for cases where account action was taken in error.
  • Appeals volume and reversal rate. What percentage of account actions are appealed and what percentage are reversed. Required under DSA; useful for procedural-fairness arguments under most regimes.
  • Moderator exposure metrics. Average hours of exposure per reviewer per week, rotation compliance, and self-reported wellbeing measures. Increasingly cited in employment litigation.
  • Categorization caveats. The Stanford CIS letter on NCMEC reporting demonstrated that approximately 80% of “Generative AI” CyberTipline reports involved no AI-generated CSAM — many were hash hits to known CSAM in AI training data. When you report aggregate numbers, document the categorization carefully (Stanford CIS).

04 · Vendor due diligence

What to ask before signing.

When you onboard a detection vendor (Thorn, Hive, NetClean, others), the diligence questions go beyond accuracy claims and pricing. Ask for evidence on:

  • Hash list source and verification chain. Where the underlying CSAM hash list comes from, who has verified it, and the refresh cadence. NCMEC, IWF, and Project Arachnid integrations are the gold standard.
  • Model training documentation. For AI classifiers, the training data sources, the validation methodology, and the false-positive / false-negative characterization on independent test sets. Public-benchmark claims should be cross-referenced.
  • Data residency and processing scope. Where your content is processed, what metadata is retained, and how long. EU and UK contracts will require GDPR-compliant processing terms; California and other state laws are converging on similar standards.
  • Security and access controls. SOC 2 Type II or equivalent; employee access logs to your content; encryption posture at rest and in transit; incident notification commitments.
  • Government request handling. The vendor's policy for responding to subpoenas, warrants, and national-security requests against the data they hold on your behalf. Transparency commitments.
  • Sunset and offboarding. Data destruction commitments, hash list access after contract end, and continuity if the vendor is acquired.

05 · Policy architecture

The internal documents that should exist.

Auditors and regulators expect a layered set of internal documents. The minimum set:

  • Acceptable Use Policy. User-facing; specific prohibitions on CSAM, NCII, sexual content involving minors, and AI-generated sexual content of minors or non-consenting adults. Plain English, multiple languages where you serve users at scale.
  • Content Moderation Policy. Internal-facing; the rules that moderators apply, with examples, edge cases, and escalation paths. Versioned; changes documented with rationale.
  • Detection-and-reporting Standard Operating Procedure. The workflow from automated detection through human review to CyberTipline reporting. Named roles, named timelines, named tools.
  • Law Enforcement Response Policy. How you handle subpoenas, warrants, emergency disclosure requests, and 2258A inquiries. Owned by counsel, documented for legal-process clarity.
  • Moderator Wellbeing Program. The exposure limits, support services, screening process, and clinical contracts. Owned jointly by HR and T&S leadership.
  • Data Protection Impact Assessment (DPIA / PIA). Required under DSA and UK GDPR for any content scanning, particularly client-side or end-to-end. Should document the privacy / safety trade-off explicitly and the proportionality analysis.
  • Transparency Reporting Standard. What you publish, on what cadence, with what level of categorization detail. Tracks both regulatory obligation (DSA quarterly) and voluntary disclosure.
  • Records Retention Schedule. Per category (reports, evidence, training records, vendor agreements, incident logs) with legal-basis-for-retention and destruction triggers.

06 · Governance and accountability

Who owns child safety, and who they answer to.

Increasingly a regulator's first question is not about your tooling but about your org chart: who is the named, accountable owner of child-safety enforcement, and where does that function report? The Australian eSafety Commissioner's mandatory transparency notices probe exactly this, and Chapter 06's comparison of TikTok, Meta, and X governance shows how differently large platforms answer it. What an assessor expects to see documented:

  • A named accountable executive. One person — by role and by name — accountable for detection, CyberTipline reporting, and child-safety policy. “The safety engineering team” is not an answer regulators accept; a platform that effectively gave that answer drew a A$650,000 Australian Federal Court penalty in May 2026 for failing to fully comply with a transparency notice.
  • A documented reporting line. An org chart showing where child safety sits and how escalation reaches the CEO. Note whether the function reports through legal: the duty to detect and report can conflict with legal's mandate to limit liability, and an assessor looks for that tension to be managed, not ignored.
  • Board-level oversight, at scale. A safety, risk, or audit committee with a documented child-safety remit, meeting minutes, and a reporting cadence. For smaller companies the equivalent is a documented founder or CEO review on a fixed schedule.
  • Continuity through turnover. The accountable role defined independently of the current officeholder, so an executive departure does not orphan the function. Document the interim-coverage plan.

This is the compliance-side mirror of the structural advice in the tech-CEO guide: the CEO makes the structural decision; compliance documents it, tests it, and can produce it on demand.

07 · Incident response

When something goes wrong.

The incidents that consume the most compliance time are not the simple known-hash-matches. They are the edge cases: a high-profile victim, a media inquiry before you've confirmed, a moderator wellbeing failure, a vendor breach, a failed report submission. A defensible incident-response framework:

  • Defined severity levels tied to specific notification obligations. Sev-1 for confirmed CSAM at scale, vendor compromise, or regulator-initiated inquiry; Sev-2 for ambiguous content involving minors or potential mandatory-reporting situations; Sev-3 for routine moderation disagreements. Each level has a named owner and a notification matrix.
  • Legal in the loop early, not late. CyberTipline reporting is the standard path, but parallel exposure (civil claims, contractual breach, international notifications) often requires counsel to weigh in within hours.
  • Communication discipline. A single source of truth for the incident timeline, a clear distinction between internal facts and external statements, and a defined no-comment threshold for active investigations. Press statements are part of the legal posture, not separate from it.
  • Post-incident review with corrective action. Documented root-cause, specific remediation items with owners and dates, and an audit trail of completion. Regulators look for this; absence of post-incident review documentation is a finding.
  • Tabletop exercises annually. Run plausible scenarios with the actual response team. Discover the gaps before the regulator does.

08 · Personnel and contractor exposure

The human-side requirements.

Beyond the technology and policy work, three personnel issues that compliance consistently has to manage:

  • Pre-hire screening for sensitive roles. Moderators, T&S investigators, and engineers with production access to user content should be screened consistent with the role's exposure. Document the screening criteria and the legal basis for them.
  • Clinical support contracts. Multiple class-action settlements (Meta, TikTok, others) have established that platform-provided clinical support for content moderators is now an industry baseline, not a benefit. Contract with specialist providers; document utilization without breaching confidentiality.
  • Contractor and BPO oversight. If you use a third-party moderation provider, the wellbeing program and exposure limits are still your compliance responsibility. The contractual passthrough is necessary but not sufficient — document the audit you perform on the provider's actual practice.

09 · If you're starting from scratch

A 90-day buildout sequence.

For compliance teams brought in late or after a regulatory inquiry, a defensible 90-day buildout:

  • Days 0–30: Audit and stabilize. Inventory existing detection coverage, reporting workflow, and policy documents. Designate a primary CyberTipline reporter today. Engage outside counsel with 2258A experience. Pull existing detection vendor agreements for review.
  • Days 30–60: Document and remediate. Write the standard operating procedures, the law-enforcement-response policy, and the moderator wellbeing program. Begin DPIA for any scanning that lacks one. Close the most visible coverage gaps (PhotoDNA enrollment, NCMEC API integration, Cloudflare scanner if applicable).
  • Days 60–90: Test and report. Run a tabletop incident exercise. Stand up the metrics dashboard. Draft the first transparency report. Brief executive leadership and the board on residual risk.

For the executive-decision side of the same exercise, see For Tech CEOs. For the courtroom side of the same evidence base, see For Attorneys.

Notes on this page

  • Informational, not legal advice. Specific compliance obligations depend on jurisdiction, entity structure, and service profile. Counsel with experience in 2258A, the DSA, and the Online Safety Act is the right escalation for entity-specific planning.
  • The vendor naming on this page is descriptive of the working ecosystem, not an endorsement. Diligence on any vendor mentioned here should still be performed independently.
  • The underlying research and policy context lives in the chapters — particularly Chapter 06 (Technology Solutions) and Chapter 07 (Prevention Strategies).