Skip to content
FORGENO ADS GAMESUnlock all $8

Responsible AI & AI Safety · Unit 3 of 8

Unit 3: Privacy and data protection

Consent, purpose limitation and data minimization, why anonymized data can often be re-identified, privacy risks special to AI, and extra care for children and sensitive data.

Opens the first lesson of this unit in Responsible AI & AI Safety. Every lesson is open from the start.

Lesson 1: Consent, purpose and minimization

Goal: Apply the core data protection principles to an AI project.

Personal data is any information about an identifiable person: a name, an email address, a face, a location history, even a device ID. Most of the world's data protection laws grew from the same handful of principles. The European Union's General Data Protection Regulation (GDPR, applying since May 2018) states them clearly, and many other laws echo them.

PrincipleIn plain wordsAI example
Lawful basisYou need a legal reason to use the data, such as consent, a contract or a legitimate interestTraining a model on customer data needs its own justification
Purpose limitationData collected for one purpose should not be quietly reused for an unrelated oneSupport chats collected to fix problems, later used to train a sales bot
Data minimizationCollect and keep only what you need for the purposeA churn model rarely needs full names or exact birth dates
AccuracyKeep personal data correct and up to dateWrong records produce wrong predictions about real people
Storage limitationDelete data when you no longer need itRaw training files kept forever "just in case"
SecurityProtect data against loss and misuseTraining data left in an open cloud folder
AccountabilityBe able to show you followed the rulesRecords of what data trained which model, and why

Consent is only one lawful basis, and under the GDPR it must be freely given, specific, informed and unambiguous, and as easy to withdraw as to give. A box buried in terms of service that says "we may use your data to improve our services" is weak ground for training AI on private messages. In the United States, as of November 2026, there is still no general federal privacy law: protection comes from sector laws (health, children, credit, education) and a growing number of state laws, led by California.

Worked example: minimizing a training set

  1. A fitness app wants an AI coach that suggests workouts. The draft dataset has name, email, date of birth, home address, GPS routes, heart rate and workout history.
  2. Ask of each field: does the coach need it? Workout history and heart rate: yes. Age: an age band (30–39) is enough, not the date of birth.
  3. Name, email and home address: not needed for training. Remove them.
  4. GPS routes reveal where people live and work. Keep distance and pace instead.
  5. Write down the purpose, tell users in plain words, and give them a real choice.

Minimization shrinks the harm if data leaks, and often makes the model no worse.

Personal data
Information about an identified or identifiable person.
GDPR
The EU General Data Protection Regulation, applying since May 2018.
Lawful basis
The legal reason that permits processing personal data, such as consent or a contract.
Consent (data protection)
A freely given, specific, informed and unambiguous agreement that can be withdrawn.
Purpose limitation
Using data only for the purposes it was collected for, or compatible ones.
Data minimization
Collecting and keeping only the personal data a purpose really needs.

Remember: Ask of every field: what is it for, do we need it, how long do we keep it, and did people really agree?

Lesson 2: Anonymization and re-identification

Goal: Explain why removing names is not anonymization, and compute k-anonymity.

Removing names and emails makes data pseudonymized, not anonymous. Other details, called quasi-identifiers, can still single people out when combined. In a widely cited 2000 study, Latanya Sweeney estimated that about 87% of the US population could be uniquely identified by just three facts: five-digit ZIP code, gender and full date of birth.

Real cases followed. In 2006, AOL released "anonymized" search logs; reporters identified a woman from her searches alone. In 2008, researchers Narayanan and Shmatikov showed that people in Netflix's "anonymous" movie-rating dataset could be re-identified by matching a few ratings with public reviews on another site. The lesson: if a record is unique, it can often be linked back to a person.

k-anonymity is a simple measure. Pick the quasi-identifiers; group records that share the same values. The dataset is k-anonymous if every group has at least k records. A group of 1 means someone is unique and exposed. You raise k by generalizing (exact age to an age band, full postcode to its first part) or suppressing rare records. k-anonymity has limits: if everyone in a group shares the same sensitive value, the group still reveals it.

Worked example: computing k for a small table

  1. Quasi-identifiers: postcode area, age band, gender. Six invented patient records: (AB1, 30–39, F), (AB1, 30–39, F), (AB1, 30–39, F), (AB2, 40–49, M), (AB2, 40–49, M), (AB3, 20–29, F).
  2. Groups: (AB1, 30–39, F) has 3 records; (AB2, 40–49, M) has 2; (AB3, 20–29, F) has 1.
  3. k is the smallest group size: k = 1. The woman in AB3 aged 20–29 is unique, so anyone who knows those three facts about her can find her diagnosis.
  4. Fix: generalize the postcode to a wider area, widen age bands, or suppress that record. After merging AB2 and AB3 into one area "AB2–3" and age into 20–49, check the groups again.

k = size of the smallest group of matching quasi-identifiers. k = 1 means someone is exposed.

Stronger tools exist. Differential privacy adds carefully calibrated random noise to results so that no single person's presence can be detected; the US Census Bureau used it for 2020 census data. It gives a mathematical guarantee, at some cost to accuracy. Synthetic data, generated to mimic real data, can help too, but it can still leak real records if the generator memorized them, so it must be tested.

Pseudonymization
Replacing direct identifiers such as names with codes. The data is still personal data.
Anonymization
Processing data so people can no longer be identified by any reasonable means.
Quasi-identifier
A detail such as postcode, age or gender that is not identifying alone but can be when combined.
Re-identification
Linking supposedly anonymous data back to the people it describes.
k-anonymity
A property where every combination of quasi-identifiers is shared by at least k records.
Differential privacy
A mathematical technique that adds calibrated noise so no individual's data can be detected in results.

Remember: No names is not anonymous. Count the smallest group of matching quasi-identifiers: that is k.

Lesson 3: Privacy risks special to AI

Goal: Recognize memorization, inference and data-leak risks in AI systems, and plan a privacy impact assessment.

AI adds privacy risks that ordinary databases do not have. Some come from what goes into a model, some from what a model can reveal, and some from what users type into AI tools.

RiskWhat it isReported example or evidence
Scraped personal dataTraining on personal data collected from the web without a lawful basisData protection authorities fined Clearview AI for building a face database from scraped photos; the Dutch authority's fine in September 2024 was about EUR 30.5 million
Memorization and extractionA model repeats pieces of its training data word for wordA 2021 study (Carlini and others) extracted names, phone numbers and addresses that GPT-2 had memorized
Membership inferenceAn attacker tests whether a specific person's record was in the training dataShown in research on many model types; revealing membership can itself be sensitive, such as being in a clinic's dataset
Inferring sensitive traitsA model guesses things people never shared, such as health or sexualityPredictions from shopping or browsing patterns
Prompt leakage to vendorsStaff paste confidential or personal data into AI toolsIn 2023 Samsung restricted generative AI tools after reports that engineers had pasted internal source code into ChatGPT
Lack of transparencyPeople are not told how their data trains or feeds AIIn December 2024 Italy's data protection authority fined OpenAI EUR 15 million over its handling of personal data in ChatGPT (OpenAI said it would appeal)

The standard tool for managing these risks is the data protection impact assessment (DPIA). Under GDPR Article 35, one is required before processing that is likely to result in a high risk to people's rights, which covers many AI uses: large-scale profiling, systematic monitoring, sensitive data, new technologies. A DPIA describes the processing and its purpose, tests whether it is necessary and proportionate, identifies risks to people, and records the measures that reduce them.

Worked example: a short DPIA for an AI meeting-notes tool

  1. Processing: the tool records meetings, transcribes them and writes summaries; the vendor hosts the model.
  2. Necessity: summaries save time; recording is needed for that, but keeping audio is not. Keep text summaries only.
  3. Risks: sensitive HR or medical details discussed in meetings; vendor training on company data; people recorded without knowing.
  4. Measures: a contract that the vendor will not train on the data; auto-delete audio after transcription; a visible "recording" notice and the right to switch it off; no recording of HR, health or legal meetings.
  5. Residual risk: low to medium; approved by the data protection officer with a review in six months.

Describe, test necessity, list risks to people, add measures, record who approved and when to review.

Memorization
A model storing pieces of its training data so it can repeat them.
Training data extraction
Getting a model to output memorized training data, which may include personal information.
Membership inference
Working out whether a particular record was used to train a model.
Data protection impact assessment (DPIA)
A documented assessment of risks to people from a data processing activity and the measures to reduce them.
Data protection officer (DPO)
The person who advises on and monitors data protection compliance in an organization.

Remember: AI can leak what it learned, reveal what it guessed, and receive what users paste. A DPIA catches these before launch.

Lesson 4: Children's data and sensitive data

Goal: Apply the extra protections for children and for sensitive categories of data.

Children cannot weigh privacy trade-offs the way adults can, and AI products aimed at them, such as tutors, games and chat companions, collect a lot. Laws therefore add protections. In the US, the Children's Online Privacy Protection Act (COPPA) requires verifiable parental consent before collecting personal information from children under 13 on services aimed at them or that know they have child users. The FTC's amended COPPA Rule (published in April 2025, with compliance required by April 22, 2026) adds, among other things, separate parental consent before disclosing children's data to third parties such as advertisers, and a written data retention policy.

In the EU, the GDPR sets 16 as the age below which a parent must consent to online services relying on consent, and lets countries lower it to as young as 13. The UK's Age Appropriate Design Code (the Children's Code, in force since September 2021) asks services likely to be used by children to put the child's best interests first, with high privacy settings by default, minimal data collection, location tracking off by default and no "nudge" techniques that push children to give up privacy. The EU AI Act also bans AI that exploits vulnerabilities due to age to distort behavior in harmful ways.

Sensitive data gets extra protection for adults too. The GDPR's "special categories" include health, genetic and biometric data used to identify someone, racial or ethnic origin, religion, political opinions, trade union membership and sex life or sexual orientation. In the US, Illinois's Biometric Information Privacy Act (2008) requires written consent before collecting face geometry or fingerprints, and lets people sue, which has led to large settlements.

Worked example: reviewing an AI homework helper for ten-year-olds

  1. Data collected: child's name, school, voice recordings, every question asked, and a "learning profile". Under COPPA this needs verifiable parental consent first.
  2. Minimize: use a nickname, not the real name; drop the school; process voice on the device or delete recordings after transcription.
  3. Defaults: chat history off for sharing, no third-party advertising, no public profile.
  4. Safety: the tutor should not ask for personal details, and conversations that suggest a child is at risk need a clear, human-led escalation process.
  5. Retention: delete the profile after a set period of inactivity, as written in a retention policy.

For children: parental consent, minimal data, safe defaults, no ads on their data, and a written retention policy.

COPPA
The US Children's Online Privacy Protection Act, requiring verifiable parental consent for collecting data from children under 13.
Verifiable parental consent
Consent from a parent obtained in a way that reasonably confirms it really is the parent.
Age Appropriate Design Code
The UK Children's Code: design standards for online services likely to be used by children.
Special category data
Under the GDPR, especially sensitive personal data such as health, biometrics, ethnicity, religion and sexual orientation.
Biometric data
Data from body measurements, such as face geometry, fingerprints or voiceprints, used to identify people.

Remember: Children and sensitive data get stricter rules: consent from the right person, less data, safe defaults, and deletion.

Practise it in Responsible AI & AI Safety

Each lesson ends with three questions that check you can use the idea, not just recognise it. Wrong answers explain why and come back in Review. The unit ends with a 7-question quiz; pass it with 80% or more to master the unit.

Every lesson is free: no ads, no lives and no paywall.