Skip to content
FORGENO ADS GAMESUnlock all $8

Data Lab · Unit 8 of 8

Unit 8: From analysis to decisions

Correlation versus causation, samples and bias, telling a clear data story and handling data ethically.

Opens the first lesson of this unit in Data Lab. Every lesson is open from the start.

Lesson 1: Correlation is not causation

Goal: Explain why two things moving together does not prove one causes the other.

Two things are correlated when they tend to move together: as one goes up, the other tends to go up (or down). It is tempting to conclude that one causes the other. Often it does not.

Worked example: ice cream and drownings

  1. In many places, ice cream sales and drownings rise and fall together through the year.
  2. Ice cream does not cause drowning.
  3. A third factor, hot weather, causes both: more people buy ice cream and more people swim.

A hidden third factor that drives both things is called a confounder.

Other explanations to always consider:

  • A confounder: something else causes both.
  • Reverse cause: Y causes X, not the other way round. Do happy employees produce good results, or do good results make employees happy?
  • Coincidence: with enough data, some things line up by pure chance.
  • Selection: the groups were different from the start.

The strongest way to test cause is an experiment: randomly split people into groups, change one thing for one group, and compare. Random assignment makes the groups alike in every other way on average, so a difference in results points to the change. Websites run these all the time as A/B tests: half of visitors see version A, half see version B.

Correlation
A tendency for two variables to move together.
Causation
When a change in one thing actually produces a change in another.
Confounder
A third factor that influences both variables, creating a correlation between them.
A/B test
An experiment that randomly shows people version A or B and compares the results.

Remember: Moving together is not proof. Look for a confounder, reverse cause, chance or selection. Experiments test cause.

Lesson 2: Samples and bias

Goal: Judge whether data represents the group you want to draw conclusions about.

You usually cannot measure everyone, so you measure a sample and generalize to the population you care about. That only works if the sample represents the population.

Sampling bias happens when some people are more likely to be included than others:

  • An online poll on a sports team's page about the team's popularity reaches its fans.
  • A customer survey answered mostly by people who are very angry or very happy.
  • A phone survey during work hours misses people who work.

Worked example: survivorship bias

A famous story from World War II: analysts studied bullet holes on bombers that returned from missions, to decide where to add armor.

  1. The returning planes had the most holes on the wings and body.
  2. The statistician Abraham Wald pointed out the problem: these were the planes that survived.
  3. Planes hit in other places, such as the engines, mostly did not come back to be counted. So the armor should go where the returning planes had no holes.

Looking only at the survivors can lead you to exactly the wrong conclusion.

Size matters too. A survey of 8 people can swing wildly by chance, while a well-chosen sample of 1,000 is much more stable. But a huge biased sample is still biased: a million responses from fans tell you about fans.

Sample
The part of a group you actually measure.
Population
The whole group you want to draw conclusions about.
Sampling bias
When the way a sample is chosen makes some members of the population more likely to be included than others.
Survivorship bias
Drawing conclusions only from the cases that "survived" a selection process.

Remember: Ask: who is in this data, and who is missing? Big and biased is still biased.

Lesson 3: Telling the story

Goal: Present findings as insight, evidence, action.

An analysis only matters if someone acts on it. A simple structure for presenting results:

  1. Insight: the main finding in one plain sentence.
  2. Evidence: the key numbers and one clear chart that prove it.
  3. Action: what you recommend, and what you expect to happen.
  4. Caveats: the limits: missing data, sample size, what you could not test.

Worked example: a three-sentence summary

Insight: Saturday afternoons bring in 31% of weekly revenue, but we have the fewest staff on then.
Evidence: Over 12 weeks, Saturday 12-5 pm averaged $1,240 vs a weekday-afternoon average of $310 (chart attached).
Action: Add one staff member on Saturday afternoons for a 4-week trial and track wait times and sales.
Caveat: Data covers summer only; winter may differ.

A busy manager can act on this in under a minute, and knows exactly what is uncertain.

Tips: lead with the answer, not the method; round numbers sensibly (31%, not 30.8742%); explain any jargon; and match the detail to the audience. Your boss wants the decision; another analyst may want the query.

Insight
A finding that changes what someone understands or decides.
Caveat
A stated limitation that tells the reader how far to trust a finding.

Remember: Insight, evidence, action, caveats. Lead with the answer.

Lesson 4: Data ethics and privacy

Goal: Handle data about people responsibly.

Much of the data analysts work with is about people. That brings responsibilities:

  • Collect only what you need. Every extra field is something that could leak or be misused.
  • Protect it. Limit who can see personal data, and never post it or share it carelessly.
  • Anonymize when you can: remove names, emails and exact addresses, or replace them with IDs, before analysis or sharing.
  • Be honest. Do not cherry-pick results, hide caveats or stretch a finding further than the data supports.
  • Consider the impact. Could a decision based on this analysis treat a group unfairly?

Laws apply too, and differ by place. For example, the European Union's GDPR sets strict rules on collecting and using personal data, and many U.S. states have their own privacy laws. If you handle customer or student data at work, ask what rules apply.

Worked example: small numbers can identify people

  1. A school publishes "average grade by class, gender and sport".
  2. One group, "girls on the wrestling team in grade 11", has exactly one student.
  3. The "average" for that group is that student's actual grade, now public.

Analysts often hide or combine groups below a minimum size, such as 5 or 10 people, so no individual can be identified.

Anonymize
Remove or replace details that identify individuals.
Personal data
Any information that can identify a specific person, alone or combined with other information.

Remember: Collect less, protect it, anonymize it and never let a small group reveal one person.

Practise it in Data Lab

Each lesson ends with two questions that check you can use the idea, not just recognise it. Wrong answers explain why and come back in Review. The unit ends with a 5-question quiz; pass it with 80% or more to master the unit.

Every lesson is free: no ads, no lives and no paywall.