Python Programming · Unit 7 of 8
Unit 7: Standard library, data and algorithms
Modules, math, random and datetime, files and io.StringIO, CSV and JSON, a data pipeline from messy export to summary, and searching, sorting and Big-O.
Opens the first lesson of this unit in Python Programming. Every lesson is open from the start.
Lesson 1: Modules: math, random and datetime
Goal: Import modules in different ways and use math, random and datetime for everyday jobs.
Python comes with a huge standard library: hundreds of modules of ready-made, well-tested code, which is why people say Python comes "with batteries included". A module is just a file of Python code. You load one with import, then use its contents with a dot.
| Form | Then use | When |
|---|---|---|
import math | math.sqrt(16) | the usual choice: it is clear where each name comes from |
from math import sqrt, pi | sqrt(16) | a few names used often |
import datetime as dt | dt.date.today() | a long module name, or a common alias |
from math import * | sqrt(16) | avoid: it hides where names come from and can overwrite yours |
math has the mathematical functions: math.sqrt(x), math.ceil(4.2) (rounds up, 5), math.floor(4.8) (rounds down, 4), math.pi, and math.isclose(a, b), the right way to compare floats. random makes pseudo-random choices: random.randint(1, 6) (both ends included), random.choice(items), random.shuffle(items) (in place) and random.sample(items, 3). Call random.seed(42) first and the "random" sequence becomes the same every run, which makes tests and simulations repeatable.
datetime handles dates and times, which are much harder than they look (month lengths, leap years, time zones). date.fromisoformat("2026-11-17") turns text into a date; subtracting two dates gives a timedelta whose .days is the gap; adding timedelta(days=30) moves a date; and strftime formats one: %Y-%m-%d gives 2026-11-17, %A the weekday name and %d %B %Y gives 17 November 2026. Never do date arithmetic by hand.
from datetime import date, timedelta
import math
issued = date.fromisoformat("2026-11-17")
due = issued + timedelta(days=30)
print(due, due.strftime("%A %d %B %Y"))
print((due - issued).days, "days")
boxes = math.ceil(53 / 12) # 53 items, 12 per box
print(boxes, "boxes")
Worked example: when is the invoice due?
fromisoformatparses the year-month-day text into a real date.- Adding 30 days crosses into December correctly: 2026-12-17.
strftime("%A %d %B %Y")writes it for people:Thursday 17 December 2026.math.ceil(53 / 12)rounds 4.42 up to 5: you cannot ship part of a box.
The datetime module knows how long every month is, so you never have to.
One warning: do not name your own file after a standard module, such as random.py or csv.py. Your file will be imported instead of the real module, with baffling errors. And when you do need something the standard library lacks, the Python Package Index (PyPI) has hundreds of thousands of third-party packages, installed with pip (Unit 8).
Hands-on lab in the room: Python playground. Run it twice: with the seed, the random values are the same each time. Delete the seed line and they change.
Hands-on lab in the room: Unit 7 coding exercises. Graded exercise: date arithmetic with datetime.
- Module
- A file of Python code you can import and reuse, such as math or csv.
- Standard library
- The collection of modules that comes with every Python installation.
- import
- The statement that loads a module so its functions and classes can be used.
- Seed
- A starting value for the random number generator; the same seed gives the same sequence.
- timedelta
- A datetime type for a length of time; subtracting dates gives one.
Remember: import module, then module.name. math.ceil rounds up; random.seed makes runs repeatable; use datetime for every date calculation.
Lesson 2: Files, CSV and JSON with io.StringIO
Goal: Explain how Python reads and writes files, use io.StringIO in their place, and read and write CSV and JSON.
Real programs read and write files. On your own computer you open a file with open(), ideally inside a with block, which closes it automatically even if an error happens. The mode says what you will do: "r" read (the default), "w" write (replacing the file), "a" append. Always give encoding="utf-8" for text so accents and symbols survive on every operating system.
# On your own computer (this page has no access to your files):
with open("orders.csv", encoding="utf-8") as f:
for line in f:
print(line.strip())
with open("report.txt", "w", encoding="utf-8") as f:
f.write("Total: 120\n")
This page runs Python in your browser, which cannot touch the files on your computer. So the labs use io.StringIO, a string that behaves exactly like an open text file: it has the same read(), readline(), write() and line-by-line looping. Code written for a StringIO works unchanged on a real file, which is why professionals use StringIO in tests too: no temporary files to create or clean up. io.StringIO(text) makes a file to read from; io.StringIO() makes an empty one to write to, and .getvalue() gives back everything written.
CSV (comma-separated values) is the most common way spreadsheets and systems export tables. Do not split lines on commas yourself: a value such as "Pens, blue" contains a comma inside quotes. The csv module handles quoting for you. csv.reader gives each row as a list of strings; csv.DictReader uses the header row to give each row as a dictionary; csv.writer and csv.DictWriter write rows back out. Every value read from CSV is a string, so convert numbers yourself.
import csv, io
text = "product,qty,price\nNotebook,3,2.5\n\"Pens, blue\",10,0.8\n"
rows = list(csv.DictReader(io.StringIO(text)))
print(rows[1]["product"], int(rows[1]["qty"]) * float(rows[1]["price"]))
out = io.StringIO()
writer = csv.writer(out)
writer.writerow(["product", "value"])
for r in rows:
writer.writerow([r["product"], int(r["qty"]) * float(r["price"])])
print(out.getvalue())
Worked example: CSV in, CSV out, through StringIO
io.StringIO(text)wraps the text so DictReader can read it like a file.- DictReader keeps
Pens, blueas one value, because it is in quotes. - Values are strings, so
int()andfloat()convert before multiplying. - The writer adds quotes around "Pens, blue" again when writing, so the output is valid CSV.
Swap io.StringIO(text) for open("orders.csv", encoding="utf-8") and the same code reads a real file.
JSON (JavaScript Object Notation) is how web APIs and many config files exchange data. It looks like Python dictionaries and lists. json.loads(text) turns JSON text into Python objects, and json.dumps(obj) turns them back into text; add indent=2 for readable output and sort_keys=True for a stable key order. (json.load(f) and json.dump(obj, f) work with files.) The types map neatly: JSON objects become dicts, arrays become lists, true/false become True/False and null becomes None. JSON has no tuples, sets or dates, so convert those first.
Hands-on lab in the room: Python playground.
Hands-on lab in the room: Unit 7 coding exercises. Graded exercise: read CSV text with csv.DictReader and io.StringIO.
- with statement
- A block that sets something up and always cleans it up, such as closing a file.
- io.StringIO
- An in-memory text file: a string you can read from and write to like a real file.
- CSV
- Comma-separated values: a plain-text table, one row per line, with quoting for commas inside values.
- csv.DictReader
- Reads CSV rows as dictionaries keyed by the header row.
- JSON
- A text format for data with objects, arrays, strings, numbers, true, false and null.
- json.loads / json.dumps
- Turn JSON text into Python objects / Python objects into JSON text.
Remember: Open files with with and encoding="utf-8". io.StringIO is a file in memory. Use the csv and json modules, and convert CSV strings to numbers.
Lesson 3: From raw data to a summary
Goal: Build a data pipeline that loads, cleans, groups, ranks and reports, handling bad rows honestly.
A huge share of junior Python work is the same job in different clothes: take an export, clean it, summarise it, report it. Sales by region, tickets per category, sign-ups per week. Building it as a pipeline of small functions, each with one job, makes it easy to test and to change.
| Stage | Job | Tools |
|---|---|---|
| Load | read the raw text into rows | csv.DictReader(io.StringIO(text)), json.loads |
| Clean | convert types, tidy labels, skip and count bad rows | int(), float(), strip().title(), try/except |
| Transform | compute new fields | revenue = qty × price |
| Aggregate | group and total | defaultdict(float), Counter |
| Report | sort, format, write out | sorted(key=...), f-strings, json.dumps |
Never drop bad data silently. Real exports have blank quantities, words where numbers should be, "n/a" prices and labels like "north " and " NORTH" that mean the same thing. Clean what you safely can (normalise case and spaces), skip what you cannot convert, and count and report what was skipped, so whoever reads the report knows how much to trust it. A summary built on 17 of 20 rows must say so.
import csv, io
from collections import defaultdict, Counter
text = """region,product,qty,price
North,Tea,4,4.25
south ,Muffin,6,2.0
East,Tea,,4.25
North,Muffin,8,2.0
"""
revenue = defaultdict(float)
units = Counter()
skipped = 0
for row in csv.DictReader(io.StringIO(text)):
try:
qty, price = int(row["qty"]), float(row["price"])
except ValueError:
skipped += 1
continue
region = row["region"].strip().title()
revenue[region] += qty * price
units[row["product"]] += qty
print(dict(revenue), units.most_common(1), "skipped:", skipped)
Worked example: the cafe pipeline in miniature
- Load: DictReader reads each line into a dictionary of strings.
- Clean: the empty quantity in the East row makes
int("")raise ValueError, so that row is counted as skipped. "south "becomes"South", so it groups with any other South rows.- Aggregate: revenue per region in a defaultdict, units per product in a Counter.
Output: {'North': 33.0, 'South': 12.0} [('Muffin', 14)] skipped: 1.
Sorting the results is part of reporting: sorted(revenue.items(), key=lambda pair: pair[1], reverse=True) ranks regions by revenue. A lambda is a tiny unnamed function written inline; lambda pair: pair[1] means "given a pair, use its second item". In the data lab you will build the full five-stage pipeline on a 20-row export, step by step, with checks at every stage.
Hands-on lab in the room: CSV-to-summary data lab. Data lab, step 2: clean the cafe export and count the bad rows. The full lab has five steps, from loading to a JSON summary.
- Pipeline
- A sequence of steps where each one's output is the next one's input: load, clean, transform, aggregate, report.
- Data cleaning
- Fixing or removing problems in raw data: wrong types, missing values, inconsistent labels.
- Aggregate
- To combine many rows into totals, counts or averages per group.
- lambda
- A small unnamed function written inline, such as lambda pair: pair[1], often used as a sort key.
Remember: Load, clean, transform, aggregate, report: one small function each. Count and report every row you skip.
Lesson 4: Searching, sorting and Big-O
Goal: Compare linear and binary search, sort with keys, and estimate how code slows down as data grows.
An algorithm is a step-by-step method for solving a problem. Two programs can give the same answer while one takes a second and the other takes an hour on big data. Big-O notation describes how the work grows as the input size n grows, ignoring small details: it is the vocabulary of every technical interview and every "why is this slow?" conversation.
Linear search checks items one by one: on average about half of them, and all of them when the item is missing. Double the list and you double the work: that is O(n). Binary search only works on sorted data: look at the middle item; if the target is smaller, discard the right half, otherwise the left half; repeat. Each look halves what is left, so a million items need only about 20 looks: O(log n).
| Big-O | Name | Example | n = 1,000 | n = 1,000,000 |
|---|---|---|---|---|
| O(1) | constant | dict lookup, x in a_set, items[i] | 1 | 1 |
| O(log n) | logarithmic | binary search | about 10 | about 20 |
| O(n) | linear | one loop over the data, x in a_list | 1,000 | 1,000,000 |
| O(n log n) | n log n | sorted() | about 10,000 | about 20,000,000 |
| O(n²) | quadratic | a loop inside a loop over the same data | 1,000,000 | 1,000,000,000,000 |
def has_duplicates_slow(items): # O(n²): compares every pair
for i in range(len(items)):
for j in range(i + 1, len(items)):
if items[i] == items[j]:
return True
return False
def has_duplicates_fast(items): # O(n): one pass with a set
seen = set()
for x in items:
if x in seen:
return True
seen.add(x)
return False
data = list(range(2000))
print(has_duplicates_slow(data), has_duplicates_fast(data))
Worked example: finding duplicates, slow and fast
- The slow version compares each item with every later item: about n²/2 comparisons, roughly 2 million for 2,000 items.
- The fast version looks at each item once and asks the set "seen before?", which is O(1).
- Both give the same answer; with a million items the slow one would take hours, the fast one under a second.
- Choosing the right data structure (a set) changed the algorithm's Big-O.
Most real speed-ups come from replacing a nested loop or a list search with a dictionary or set.
Sorting. Python's sorted() and list.sort() use an efficient O(n log n) algorithm, so do not write your own sort at work; do understand one, such as selection sort (repeatedly pick the smallest remaining item, O(n²)), because interviews ask. Sort by any rule with key=: sorted(words, key=len), sorted(people, key=lambda p: p["age"]), and sort by several rules with a tuple, key=lambda p: (-p["score"], p["name"]) (score high to low, then name A to Z). Python's sort is stable: items that compare equal keep their original order.
Hands-on lab in the room: Python playground. The last lines time the same search in a list and in a set. Exact times vary by device; the gap does not.
Hands-on lab in the room: Unit 7 coding exercises. Graded exercise: binary search (a check counts how many items you look at).
- Algorithm
- A step-by-step method for solving a problem.
- Big-O notation
- A description of how an algorithm's work grows with input size n, such as O(n) or O(log n).
- Binary search
- Searching sorted data by repeatedly halving the range: O(log n).
- Quadratic time
- O(n²): work grows with the square of the input, typical of a loop inside a loop.
- Stable sort
- A sort that keeps equal items in their original order.
- Sort key
- A function given to sorted(key=...) that says what to sort each item by.
Remember: Linear search is O(n); binary search on sorted data is O(log n); a dict or set lookup is O(1). Nested loops over the same data are O(n²).
Practise it in Python Programming
Each lesson ends with three questions that check you can use the idea, not just recognise it. Wrong answers explain why and come back in Review. The unit ends with a 7-question quiz; pass it with 80% or more to master the unit. Hands-on lab: CSV-to-summary data lab and Unit 7 coding exercises.
Every lesson is free: no ads, no lives and no paywall.