AI Data Annotation & Model Rating · Career skills · Full course
Free AI data annotation and model rating course

AI Data Annotation & Model Rating is a free, full career course for remote AI data work as a data annotator, AI trainer or search and AI rater. It has 8 units and 32 lessons: why AI needs human-labelled data, classification, bounding boxes, polygons, entity spans, sentiment, transcription and ranking, reading long guidelines and handling edge cases and escalation, gold questions, audits, percent agreement and Cohen's kappa by hand, rating AI responses for helpfulness, honesty, harmlessness and instruction-following, side-by-side preferences and how RLHF uses them, writing ideal responses and fact-checking them, judging code, safety labelling and protecting your wellbeing, confidentiality, speed versus quality, how the contract work and its taxes work in general terms, job-scam safety and building a profile.
Every lesson explains one idea in plain words, works a real example step by step, then asks two or three questions that make you use it. Wrong answers explain why and come back in Review until they stick. There are 152 practice and quiz questions in all, plus eight hands-on labs: bounding-box labeller, entity span tagger, guideline edge-case lab, agreement and kappa calculator, rubric rating and calibration lab, side-by-side response rater, fact-check desk and scam-or-legit job sorter.
Pass every unit quiz with 80% or more and finish the final project, running a mini annotation project with a guideline, an agreement check, side-by-side ratings and a fact-checked response, to earn a printable certificate of completion. Progress is saved only in this browser on this device, with nothing to sign up for.
Every lesson is free: no ads, no lives and no paywall.
What you'll learn
- How models learn from labelled examples, the jobs that create those labels, where a label goes after you submit it, and what quality really means.
- Classification and multi-label tags, boxes, polygons and segmentation, entity spans, sentiment, transcription and ranking, with the exact rules each one needs.
- How to read and apply a long guideline precisely, handle edge cases without guessing, and escalate the right things the right way.
- Gold questions, audits and feedback; percent agreement and why it can mislead; Cohen's kappa worked by hand; and how teams raise agreement.
- The four things raters judge, 1 to 7 scales and calibration, side-by-side preferences, and how your comparisons train a model through RLHF.
- Writing ideal responses, editing model answers and writing test prompts, fact-checking claims against sources, and the basics of judging a coding response.
- Labelling against a safety policy, handling disturbing material while protecting your wellbeing, keeping task data confidential, and balancing speed with quality.
- How remote AI data work really works, contractor money basics, how to spot and report job scams, and how to build a profile, a portfolio and your next step.
All 8 units
- Unit 1Why AI needs human labelsModels learn from labelled examples · The jobs: annotator, AI trainer, rater · Where your label goes · What quality means
- Unit 2Annotation typesClassification: one label or many · Boxes, polygons and segmentation · Named entities and text spans · Sentiment, transcription and ranking
- Unit 3Guidelines, edge cases and escalationReading a long guideline · Applying rules precisely · Edge cases: never guess · When and how to escalate
- Unit 4Consistency and agreementGold questions, audits and feedback · Percent agreement and its trap · Cohen's kappa, by hand · Raising agreement
- Unit 5Rating AI responsesThe four things raters judge · Rating scales and calibration · Side-by-side preference ranking · How your preferences train a model
- Unit 6Writing, fact-checking and code reviewWhat an ideal response looks like · Editing responses and writing prompts · Fact-checking with sources · Evaluating code responses
- Unit 7Safety, wellbeing and confidentialitySafety labelling · Disturbing content and your wellbeing · Privacy, confidentiality and NDAs · Speed versus quality
- Unit 8The job: platforms, money, scams and your profileHow remote AI data work really works · Contractor money basics · Job-scam safety · Your profile, portfolio and next step
Questions
Is AI Data Annotation & Model Rating really free?
Yes. Every lesson is free: no ads, no lives and no paywall. There is no account to make, and your progress is saved in your own browser.
What jobs does it prepare me for?
Entry-level remote AI data work: data annotator or labeller, AI trainer or tutor, and search, ads or AI response rater. The last unit explains how these contract jobs work, how to spot fake job offers, and the next roles to aim for, such as reviewer or QA and specialist trainer.
Does it tell me how much the work pays?
No. Pay varies by project, skill, language and country, and it changes over time, so the course never states rates. It shows you how to read a project's current terms, and reminds you that legitimate work never asks you to pay to start.
Do the labs use real data or a real AI?
No. The eight labs run in your browser on invented material: a bounding-box labeller scored with IoU, an entity tagger, a guideline edge-case lab, a kappa calculator, a rubric rating lab, a side-by-side rater, a fact-check desk and a scam-or-legit job sorter. The second rater and expert judgments are written examples.
Is the certificate an industry certification?
No. It shows you completed every lesson, passed every unit quiz and finished the project of a free Forge course. It is not an official industry certification. The project, a mini annotation project with an agreement check, is portfolio evidence in its own right.