AI Skills Academy · Unit 4 of 8
Unit 4: Checking AI's work
Hallucinations, verifying claims, known weak spots and bias: the skills that separate careful AI users from careless ones.
Opens the first lesson of this unit in AI Skills Academy. Every lesson is open from the start.
Lesson 1: Hallucinations: confident and wrong
Goal: Recognize hallucinations and the situations where they are most likely.
When an AI produces something that sounds right but is false or made up, it is called a hallucination. It is not lying on purpose. It is doing what it was built to do, generating plausible text, in a situation where plausible and true have come apart.
Hallucinations are most common for things that have a specific "shape" the model can imitate: citations, quotes, statistics, book titles, web addresses, court cases, product specs, and biographies of people who are not famous.
Worked example: a real case with real consequences
In 2023, lawyers in a New York federal court case (Mata v. Avianca) filed a legal brief that cited court decisions found with ChatGPT.
- Several of the cited cases did not exist. The AI had produced realistic-looking case names, quotes and citations.
- When the court asked for copies, the cases could not be found.
- The judge fined the lawyers and their firm $5,000 and required them to notify the judges who had been falsely named as authors of the fake opinions.
The AI produced the fakes, but the people who filed them without checking were the ones held responsible.
- A very precise number with no source ("71.3% of teens...")
- A quote that is suspiciously perfect for your point
- A source, link or book you cannot find anywhere
- An answer to something it could not know, such as private or very recent information
- Hallucination
- AI output that sounds plausible but is false or made up, such as invented facts, quotes or sources.
Remember: The more specific a claim (a number, a name, a quote, a citation), the more it needs checking.
Lesson 2: How to verify a claim
Goal: Use lateral reading and source checks to confirm or reject a claim.
Professional fact-checkers use a habit called lateral reading: instead of staring harder at the claim, they leave it and open new tabs to see what independent, reliable sources say.
- Find the original source. For a statistic, find the study or the official agency that published it. For a quote, find the speech or the article.
- Look for two independent reliable sources. Two sites copying the same blog post count as one.
- Check the date. Is the information current, or has it changed?
- If the AI gives a source, open it. Does it exist? Does it actually say what the AI claimed it says?
What counts as reliable depends on the claim: government agencies for official statistics, the organization itself for its own prices and policies, peer-reviewed research and established news organizations with corrections policies for news and science.
Hands-on lab in the room: Fact-check desk. An AI wrote an answer about an invented town's bridge. Use the source cards to decide which claims hold up.
Three possible results: the claim is supported, it is contradicted, or you cannot verify it. "Cannot verify" is a real result. It means do not repeat it as fact.
- Lateral reading
- Checking a claim by leaving it and seeing what other independent, reliable sources say about it.
- Primary source
- The original source of information, such as the study, the dataset, the official document or the person who said it.
Remember: Leave the answer, open new tabs, find the original source. "Cannot verify" means do not repeat it.
Lesson 3: Known weak spots
Goal: Anticipate the specific places AI tends to fail, and test for them.
Some AI failures are random. Many are predictable, and knowing them makes you much harder to fool.
| Weak spot | What goes wrong | What to do |
|---|---|---|
| Arithmetic and counting | Wrong totals, miscounted letters or items | Check with a calculator or spreadsheet. Some apps run real code for math; check it anyway. |
| Recent or local facts | Out of date or invented | Go to the source or a current reliable site. |
| Exact quotes and citations | Paraphrased or invented | Find the original. |
| Agreeing with you | Changes a correct answer when you push back, or praises a bad idea | Ask for evidence, or ask it to argue the opposite side. |
| Its own abilities | Claims it did something it cannot do, like "I checked the website" | Ask how it knows. Look for a shown source. |
Worked example: the "are you sure?" trap
- You ask a math question and the AI answers 42, which is correct.
- You reply "Are you sure? I think it is 48."
- Many models will apologize and switch to 48, because agreeing with the user is a pattern they learned. This tendency is called sycophancy.
Instead of "are you sure?", ask: "Show the steps." Then check the steps yourself.
- Sycophancy
- An AI tendency to agree with or flatter the user, even when the user is wrong.
Remember: Do not ask "are you sure?" Ask "show me the steps" or "what is the evidence?"
Lesson 4: Bias in, bias out
Goal: Explain where AI bias comes from and run simple tests for it.
AI learns from data made by people, and that data contains people's biases and gaps. So AI can repeat stereotypes (always picturing nurses as women and engineers as men) or work less well for groups that were underrepresented in its training data.
Worked example: a hiring tool that learned the wrong lesson
Reuters reported in 2018 that Amazon had stopped using an experimental AI tool for rating job applicants.
- The tool learned from resumes the company had received over about ten years, most of them from men.
- It learned to favor patterns from those past hires, and reportedly downgraded resumes that included the word "women's", as in "women's chess club captain".
- Engineers tried to fix it, but could not be sure it would not find other unfair patterns, and the project was dropped.
Nobody told the tool to discriminate. It copied a pattern that was already in its data.
Two simple tests you can run yourself:
- Swap test: ask the same question twice, changing only a name, gender, age or neighborhood. If the answers change in ways that should not matter, you have found bias.
- Range test: ask for "a picture of a scientist" or "a story about a CEO" several times. Who keeps showing up? Who never does?
When the stakes are real, like hiring, school discipline, loans or healthcare, people should be asking hard questions about how the AI was trained and tested, and there should always be a way for a human to review a decision.
Remember: AI can copy the biases in its data. Swap one detail and see if the answer changes.
Practise it in AI Skills Academy
Each lesson ends with two or three questions that check you can use the idea, not just recognise it. Wrong answers explain why and come back in Review. The unit ends with a 5-question quiz; pass it with 80% or more to master the unit. Hands-on lab: Fact-check desk.
Every lesson is free: no ads, no lives and no paywall.