Skip to content
FORGENO ADS GAMESUnlock all $8

Responsible AI & AI Safety · Unit 6 of 8

Unit 6: Deepfakes, synthetic media and trust & safety

How deepfakes cause harm and the laws aimed at them, C2PA Content Credentials and watermarks, verifying media without trusting detectors, and trust and safety at scale.

Opens the first lesson of this unit in Responsible AI & AI Safety. Every lesson is open from the start.

Lesson 1: How synthetic media causes harm

Goal: Name the main harms of deepfakes and the laws aimed at them.

Synthetic media is images, audio, video or text generated or altered by AI. Most of it is harmless or useful: dubbing, accessibility, art, product photos. A deepfake is synthetic media that realistically shows a real person saying or doing something they did not. The harms cluster into a few kinds.

HarmReported example
Fraud and impersonationFebruary 2024: Hong Kong police reported a finance worker sent about US$25 million after a video call with deepfaked executives (the firm was later named as Arup). Families are also targeted with cloned voices of relatives "in trouble".
Election manipulationJanuary 2024: robocalls with a cloned voice of President Biden told New Hampshire voters not to vote in the primary. In February 2024 the FCC ruled that AI-generated voices in robocalls count as "artificial" under existing law, and it later fined the political consultant behind the calls $6 million.
Non-consensual intimate imagerySexual deepfakes of real people, overwhelmingly women and including minors, are the most common harmful deepfakes. "Nudify" apps made them easy to produce.
The liar's dividendAs fakes become common, people can dismiss real evidence as "probably AI", which weakens trust in genuine recordings.

Laws have moved fastest on intimate imagery. The US TAKE IT DOWN Act, signed in May 2025, made publishing non-consensual intimate images, including AI-generated ones, a federal crime, and requires covered platforms to remove them within 48 hours of a valid request; those platform duties took effect on May 19, 2026, enforced by the FTC. In the EU, the Digital Omnibus on AI (Regulation (EU) 2026/1744) adds a ban on AI systems that generate non-consensual intimate imagery or child sexual abuse material, applying from 2 December 2026. EU AI Act Article 50 requires labeling of deepfakes from 2 August 2026. Many US states have their own deepfake laws on elections and intimate images.

Worked example: a family voice-clone scam, and the defense

  1. A grandparent gets a call: their grandson's voice, crying, says he has been arrested and needs bail money sent now, and asks them not to tell his parents.
  2. Red flags: urgency, secrecy, an unusual payment method, and a voice that could be cloned from a few seconds of social media video.
  3. Defense: hang up and call the grandson or his parents on a number you already know. Families can agree a safe word in advance that a real caller would know.
  4. The same pattern protects companies: verify payment requests through a second, known channel, however convincing the voice or video.

Do not try to out-listen the fake. Verify through a channel the attacker does not control.

Synthetic media
Images, audio, video or text generated or substantially altered by AI.
Deepfake
Synthetic media that realistically depicts a real person saying or doing something they did not.
Voice cloning
Using AI to imitate a specific person's voice from samples of their speech.
Non-consensual intimate imagery (NCII)
Sexual images of a person shared without their consent, including AI-generated ones.
Liar's dividend
The ability to dismiss real evidence as fake because fakes are known to exist.
TAKE IT DOWN Act
A 2025 US federal law criminalizing non-consensual intimate imagery, including deepfakes, and requiring platforms to remove it within 48 hours of a valid request.

Remember: Fraud, elections, intimate imagery and the liar's dividend: verify through a channel the attacker does not control.

Lesson 2: Content provenance, watermarks and C2PA

Goal: Explain what Content Credentials and watermarks can and cannot prove.

There are two broad approaches to telling real from synthetic. Detection looks at the content and guesses. Provenance records where content came from and what happened to it, so you can check rather than guess. Detection is an arms race that detectors tend to lose as generators improve; provenance is more promising but only works where it has been adopted.

The leading provenance standard comes from the Coalition for Content Provenance and Authenticity (C2PA), founded in 2021 by companies including Adobe, Microsoft, Intel, Arm, the BBC and Truepic. Its Content Credentials are signed records (manifests) attached to a file, stating which device or software made it, what edits were applied and whether AI was used. Each manifest is cryptographically signed and tied to the file's contents, so tampering can be detected. Version 2.4 of the specification was released in April 2026. Adopters include camera makers (Leica, Sony and Canon have shipped support in some models; Nikon suspended its service in 2025 after a flaw was found), Google's Pixel 10 phones (2025), and AI image generators such as OpenAI's; LinkedIn and TikTok display credentials.

Watermarks are signals hidden in the content itself (for example Google's SynthID), which can survive some editing where attached metadata would be stripped. The EU AI Act (Article 50) requires providers of generative AI to mark outputs in a machine-readable way from 2 August 2026 (systems already on the market before then have until 2 December 2026), and the European Commission's final Code of Practice on marking and labelling AI-generated content, published in June 2026, recommends combining more than one technique because no single method is enough.

What provenance can tell youWhat it cannot tell you
Which device or tool signed the file, and whenWhether the scene shown is true or presented in honest context
What edits were recorded after thatAnything about a file whose credentials were stripped
Whether the file changed after signingThat a file without credentials is fake (most real photos have none)

Worked example: reading a content credential

  1. A photo carries a credential: captured on a camera model with C2PA support, signed by the camera maker, then cropped and colour-adjusted in an editing app, with no AI generation recorded.
  2. The signature checks out and the file has not changed since its last signed edit.
  3. What this supports: the photo came from that camera and was only lightly edited.
  4. What it does not show: where or when the scene really took place, or whether the caption describes it honestly. An old, real photo can be re-posted with a false caption, credentials and all.
  5. Security researchers reported weaknesses in some implementations in 2025 and 2026 (Nikon withdrew its camera certificates in 2025 after one), so treat credentials as strong evidence, not unbreakable proof.

A valid credential is strong evidence of origin and edits. It is not proof that a story is true, and a missing one proves nothing.

Content provenance
A record of where a piece of content came from and how it was changed.
C2PA
The Coalition for Content Provenance and Authenticity, which publishes the Content Credentials standard.
Content Credentials
Cryptographically signed provenance records attached to a file under the C2PA standard.
Digital watermark
A hidden signal embedded in content to mark its origin, designed to survive some editing.
Metadata stripping
The removal of embedded information such as credentials, often done automatically when files are uploaded or compressed.

Remember: Provenance shows origin and edits when present. A missing credential proves nothing, and a valid one does not prove the caption.

Lesson 3: Verifying media without trusting detectors

Goal: Use a repeatable verification method and reach an honest verdict.

You will rarely be able to prove a piece of media is fake by looking at it. Six fingers and melted text were common clues in early AI images; current tools rarely make those mistakes, and real photos can have odd artifacts too. Professional fact-checkers verify the claim and the source, not the pixels.

A widely taught method is SIFT, developed by digital-literacy researcher Mike Caulfield: Stop (notice your reaction before sharing); Investigate the source (who posted it first, and what is their track record?); Find better coverage (do reliable outlets report the same thing?); Trace claims, quotes and media to their original context (reverse image search often finds an older original or the real location).

  • Provenance: are there Content Credentials? Do they check out?
  • Source: who first posted it, and are they who they claim to be?
  • Corroboration: do independent, reliable sources confirm it?
  • Context: does a reverse image or video search find it earlier, elsewhere, or with a different caption?
  • Plausibility: does it match what is known about the place, time, weather and people?
  • Detector results: at most a weak hint, never a verdict.

AI-content detectors are unreliable in both directions: they miss fakes and flag real content, and scores can change after simple edits such as compression or cropping. They are especially poor on audio and on short clips. Never accuse anyone, remove content or publish a verdict on a detector score alone. The honest verdict is often "unverified": not confirmed real, not shown to be fake, and not to be shared as fact.

Worked example: the video of a flooded airport

  1. A video of a flooded airport terminal is spreading with the caption "happening now at our city's airport".
  2. Stop: it is dramatic and designed to be shared.
  3. Source: posted by a new account with no history.
  4. Trace: a reverse search of a frame finds the same video posted three years earlier, captioned with a different airport in another country.
  5. Verdict: real footage, false context. It is misleading even though no AI was involved, and a deepfake detector would have called it "real".

Most misleading media is real content in false context. Checking context catches what detectors never will.

Hands-on lab in the room: Synthetic media provenance checklist. Work through invented cases with the provenance checklist and reach a verdict for each: verified, misleading or false, or unverified. The lab never treats a detector score as decisive.

SIFT
A verification method: Stop, Investigate the source, Find better coverage, Trace to the original context.
Reverse image search
Searching with an image to find where else, and when, it has appeared.
Out of context
Real media presented with a false time, place or meaning.
AI-content detector
A tool that guesses whether content is AI-generated. Unreliable in both directions.
Unverified
Not confirmed as genuine or as fake; should not be shared as fact.

Remember: Verify the source and the context, not the pixels. "Unverified" is an honest answer; a detector score is not a verdict.

Lesson 4: Trust and safety at scale

Goal: Describe how platforms write policy, enforce it at scale with AI and people, and measure the results.

Trust and safety (T&S) teams keep online platforms usable and safe: they write the rules (community guidelines), enforce them, handle appeals and report on what they did. Synthetic media is now a large part of the work, alongside spam, harassment, scams, child safety and violent extremism.

The work is a pipeline. Policy defines what is not allowed, with examples and edge cases (for instance, satire versus impersonation). Detection combines user reports, automated classifiers, hash-matching of known abusive images, and provenance signals. Enforcement chooses an action proportionate to the harm: remove, label, reduce distribution, age-restrict, or suspend an account. Appeals let people challenge mistakes. Transparency reports publish the numbers. In the EU, the Digital Services Act (applying in full since February 2024) requires platforms to explain moderation decisions to users and very large platforms to assess systemic risks.

Automation is essential at scale and imperfect, so T&S teams watch error rates closely. A classifier's precision says how many removals were right; its recall says how much violating content it caught. Raising one usually lowers the other, and the right balance depends on the harm: child safety demands very high recall with fast human review, while borderline speech calls for high precision and generous appeals.

Worked example: reading a moderation classifier's numbers

  1. An invented platform's deepfake-impersonation classifier auto-removed 10,000 posts last week.
  2. Reviewers sampled them and estimate 9,200 really broke the rules: precision = 9,200 ÷ 10,000 = 92%.
  3. So about 800 legitimate posts were wrongly removed in a week. Appeals capacity must handle that.
  4. Separately, reviewers estimate 2,000 violating posts were missed: recall = 9,200 ÷ (9,200 + 2,000) ≈ 82%.
  5. Decision: keep automatic removal only for the highest-confidence cases; send the middle band to human review; publish error rates in the transparency report.

Precision tells you how many people were wronged by removals; recall tells you how much harm got through.

People do this work too. Human moderators review the hardest and most disturbing content, so responsible T&S programs include wellbeing support, exposure limits and fair working conditions. Policy also needs care for free expression: over-removal silences legitimate speech, and errors often fall hardest on minority languages and communities that classifiers handle worst.

Trust and safety (T&S)
The teams and practices that keep online platforms safe: policy, enforcement, appeals and transparency.
Community guidelines
A platform's published rules about what content and behavior are allowed.
Hash-matching
Detecting known abusive images by comparing digital fingerprints with a database of confirmed examples.
Transparency report
A published report of a platform's moderation actions, error rates and government requests.
Digital Services Act (DSA)
An EU law setting duties for online platforms on illegal content, transparency and systemic risk.

Remember: Policy, detection, proportionate enforcement, appeals and transparency, with precision and recall watched by harm type.

Practise it in Responsible AI & AI Safety

Each lesson ends with three questions that check you can use the idea, not just recognise it. Wrong answers explain why and come back in Review. The unit ends with a 7-question quiz; pass it with 80% or more to master the unit. Hands-on lab: Synthetic media provenance checklist.

Every lesson is free: no ads, no lives and no paywall.