Machine Learning Fundamentals · Unit 2 of 8
Unit 2: Data, features and labels
Examples, features and labels; turning categories and dates into numbers; scaling features; and the messy-data problems, especially leakage, that sink real projects.
Opens the first lesson of this unit in Machine Learning Fundamentals. Every lesson is open from the start.
Lesson 1: Examples, features and labels
Goal: Identify the examples, features and label in a dataset.
Data for machine learning is usually a table. Each row is one example: one house, one email, one customer. Each column is a feature, an input the model can use, except one: the label, the answer you want to predict.
| size_m2 | bedrooms | garden | distance_km | sale_price |
|---|---|---|---|---|
| 85 | 2 | no | 3.2 | 210,000 |
| 120 | 3 | yes | 7.5 | 265,000 |
| 64 | 1 | no | 1.1 | 198,000 |
| 150 | 4 | yes | 12.0 | 290,000 |
Choosing features is where knowledge of the business matters. A good feature plausibly relates to the label, is available at the moment you need to predict, and is measured reliably. The model can only find patterns in what you give it; it cannot invent information that is not in the table.
You will often see the features of one example written as x and its label as y. Training finds a function f so that f(x) is close to y for the training examples, and, more importantly, for new ones.
- Example (row)
- One item in the dataset, such as one house or one customer.
- Feature
- An input column the model can use to make its prediction.
- Label (target)
- The column the model learns to predict.
Remember: Rows are examples, columns are features, and one column is the label. Features must exist at prediction time.
Lesson 2: Turning data into numbers
Goal: Encode categories, yes/no values and dates as features.
Models work with numbers, so every feature has to become numbers.
| Feature type | Example | How it becomes numbers |
|---|---|---|
| Numeric | Size 85 m², age 34 | Use as is (often scaled: Lesson 3) |
| Yes or no | has_garden | 1 for yes, 0 for no |
| Category | City: Leeds, York, Hull | One-hot: a 0/1 column for each city |
| Ordered category | Size: S, M, L | 1, 2, 3, because the order means something |
| Date | 2026-11-17 | Useful parts: day of week, month, days since signup |
| Text | An email body | Word counts or embeddings (Unit 8) |
Worked example: one-hot encoding
- City has three values: Leeds, York and Hull.
- Coding them 1, 2 and 3 tells the model that Hull is "three times" Leeds and that York sits between them, which is nonsense.
- One-hot encoding uses three columns instead: city_leeds, city_york and city_hull. A house in York becomes 0, 1, 0.
Use plain numbers only when the order and the distances between values mean something.
Identifiers such as customer numbers or order IDs are not features, even though they are numbers: customer 50,412 is not "more" than customer 50,411. Leave them out, or the model may learn coincidences, such as newer customers having higher IDs.
- One-hot encoding
- Turning a category into one 0/1 column per possible value.
- Categorical feature
- A feature whose values are categories rather than quantities.
Remember: Categories become one-hot columns, yes/no becomes 1/0, dates become useful parts. IDs are not features.
Lesson 3: Feature scaling
Goal: Scale features with min-max scaling and standardization, using training data only.
Some methods, including k-nearest neighbours (Unit 3) and gradient descent (Unit 4), are sensitive to the size of the numbers. If one feature is income in dollars (20,000 to 200,000) and another is age in years (18 to 90), a difference of 1,000 dollars looks a thousand times bigger than a difference of one year, even if age matters more.
Scaling puts features on comparable ranges. Two common methods:
- Min-max scaling: (x − min) ÷ (max − min). The smallest value becomes 0 and the largest becomes 1.
- Standardization: (x − mean) ÷ standard deviation. The feature is centered on 0, and most values fall between about −2 and 2.
Worked example: min-max scaling by hand
- Ages in the training data run from 18 to 90. A 54-year-old becomes (54 − 18) ÷ (90 − 18) = 36 ÷ 72 = 0.5.
- Incomes run from 20,000 to 200,000. An income of 65,000 becomes (65,000 − 20,000) ÷ 180,000 = 0.25.
- Now a step of 0.1 in either feature means one tenth of its range.
Compute the min, max, mean and standard deviation on the training data only, then apply the same numbers to test data and new inputs.
Watch out: Outliers distort min-max scaling: one income of 5,000,000 squashes everyone else close to 0. Standardization, or capping extreme values first, copes better.
- Feature scaling
- Transforming features onto comparable ranges so that no feature dominates just because of its units.
- Min-max scaling
- Rescaling a feature so that its training minimum is 0 and its maximum is 1.
- Standardization
- Rescaling a feature to mean 0 and standard deviation 1.
Remember: Scale with numbers from the training data only, then reuse them everywhere else.
Lesson 4: Messy data and leakage
Goal: Handle missing values and duplicates, and spot leakage before it fools you.
Real data is messy: missing values, duplicates, typing errors and labels that are simply wrong. Common fixes are to remove exact duplicates, fill missing numbers with the training median (and add a "was missing" flag if the gap itself might mean something), and check a sample of labels by hand. Garbage in, garbage out.
Leakage is the most dangerous mistake in machine learning: a feature that secretly contains the answer, or information that would not be available when you make a real prediction. The model looks brilliant in testing and fails in real use.
Worked example: a leak that looked like genius
- A team predicts which customers will complain. The model is 99% accurate.
- The most important feature is "refund_issued". Refunds are issued after complaints, so the feature contains the answer.
- At prediction time, before any complaint, no refund exists yet. Without that feature the model is 71% accurate, its real performance.
If a result looks too good to be true, look for leakage.
| Leaky feature | Why it leaks |
|---|---|
| refund_issued, to predict complaints | It happens after the event |
| Average delivery time including the order being predicted | It uses the answer |
| The treatment given, to predict the diagnosis | Treatment follows diagnosis |
| A random split of data recorded over time | Future rows help predict past ones (Unit 3) |
- Leakage
- When information that would not be available at prediction time, often the answer itself, gets into the features or training process.
- Missing value
- A blank or unknown entry, which must be handled before training.
Remember: Ask of every feature: would I really know this at the moment of prediction?
Practise it in Machine Learning Fundamentals
Each lesson ends with three questions that check you can use the idea, not just recognise it. Wrong answers explain why and come back in Review. The unit ends with a 6-question quiz; pass it with 80% or more to master the unit.
Every lesson is free: no ads, no lives and no paywall.