Comprehensions - Python Comprehensions and Generators

10.0 What This Teaches

This tutorial covers Python's concise collection-building syntax:

List comprehensions: [expr for item in iterable]
Filtering with if conditions
Nested comprehensions for multi-dimensional data
Dict and set comprehensions
Generator expressions: lazy evaluation with parentheses
When to use a comprehension vs a regular loop

10.1 List Comprehensions

A list comprehension builds a new list by applying an expression to each element of an iterable. The form is [expression for variable in iterable]:

squares = [x ** 2 for x in range(1, 6)]
print(squares)   # [1, 4, 9, 16, 25]

words = ["hello", "world", "python"]
upper = [w.upper() for w in words]
print(upper)     # ['HELLO', 'WORLD', 'PYTHON']

lengths = [len(w) for w in words]
print(lengths)   # [5, 5, 6]

Compared to an explicit for loop with .append(), comprehensions are more concise and typically slightly faster because the list is built in one step.

10.2 Filtering with if

Add an if clause after the for clause to include only elements that satisfy a condition:

numbers = range(20)
evens = [n for n in numbers if n % 2 == 0]
print(evens)   # [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]

words = ["apple", "fig", "banana", "kiwi", "cherry"]
long_words = [w for w in words if len(w) > 4]
print(long_words)   # ['apple', 'banana', 'cherry']

The condition is not a ternary expression - it filters which items enter the comprehension at all. A ternary in the expression part transforms items: [x if x > 0 else 0 for x in values] (clamp negatives to 0).

10.3 Nested Comprehensions

# Flatten a matrix
matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
flat = [n for row in matrix for n in row]
print(flat)   # [1, 2, 3, 4, 5, 6, 7, 8, 9]

# Cartesian product
pairs = [(x, y) for x in range(1, 4) for y in range(1, 4) if x != y]
print(pairs[:4])   # [(1, 2), (1, 3), (2, 1), (2, 3)]

# 3x3 identity matrix
identity = [[1 if i == j else 0 for j in range(3)] for i in range(3)]
print(identity)

Read nested comprehensions left to right: the outer for is listed first, exactly as it would appear in nested for loops. Prefer regular loops when nesting gets three or more levels deep.

10.4 Dict Comprehensions

words = ["apple", "banana", "cherry"]
word_lengths = {w: len(w) for w in words}
print(word_lengths)   # {'apple': 5, 'banana': 6, 'cherry': 6}

# Invert a dict (assumes values are unique)
grades = {"Alice": "A", "Bob": "B", "Carol": "A"}
inverted = {v: k for k, v in grades.items()}
print(inverted)   # {'A': 'Carol', 'B': 'Bob'}

# Conditional - keep only passing grades
passing = {name: g for name, g in grades.items() if g != "F"}
print(passing)

10.5 Set Comprehensions

sentence = "the quick brown fox jumps over the lazy dog"
letters = {ch for ch in sentence if ch.isalpha()}
print(sorted(letters))   # every unique letter in the alphabet (26)
print(len(letters))      # 26 - the pangram contains every letter

words = ["Cat", "cat", "Dog", "dog", "Bird"]
unique_lower = {w.lower() for w in words}
print(unique_lower)   # {'cat', 'dog', 'bird'}

Set comprehensions use curly braces like dict comprehensions but produce a set (no duplicate values, unordered). Use them when you need uniqueness.

10.6 Generator Expressions

Replace square brackets with parentheses to get a generator instead of a list. Generators are lazy - they produce one value at a time and don't build the entire collection in memory:

# List comprehension - builds all million values in memory
squares_list = [x ** 2 for x in range(1_000_000)]

# Generator expression - computes one value at a time
squares_gen = (x ** 2 for x in range(1_000_000))

# sum() accepts any iterable - generator is fine here
total = sum(x ** 2 for x in range(1001) if x % 2 == 0)
print(total)   # sum of squares of even numbers 0..1000

# any() / all() short-circuit on generators
has_negative = any(x < 0 for x in [1, 2, -3, 4])
print(has_negative)   # True (stops at -3)

Pass a generator directly to functions like sum, max, min, any, all, and join. Don't wrap in a list unless you need to iterate it multiple times.

10.7 Example - All Together

# Comprehensions - word frequency, matrix transpose, and prime sieve.

# Word frequency dict
text = "to be or not to be that is the question to be"
freq = {word: text.split().count(word) for word in set(text.split())}
top3 = sorted(freq.items(), key=lambda kv: kv[1], reverse=True)[:3]
print(top3)

# Matrix transpose with nested comprehension
matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
transposed = [[row[i] for row in matrix] for i in range(3)]
print(transposed)

# Sieve of Eratosthenes - primes up to 50
limit = 50
composites = {j for i in range(2, int(limit**0.5) + 1) for j in range(i*2, limit+1, i)}
primes = [n for n in range(2, limit + 1) if n not in composites]
print(primes)

Expected output:

[('be', 3), ('to', 3), ('or', 1)]
[[1, 4, 7], [2, 5, 8], [3, 6, 9]]
[2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47]

10.8 Exercise

Exercise

Use a list comprehension to generate the first 20 Fibonacci numbers (hint: use a helper function or start from a seed list).
Use a dict comprehension to map each integer from 1 to 10 to its cube.
Use a generator expression inside sum() to compute the sum of all integers from 1 to 1 000 000 that are divisible by 3 or 5. Compare the result to the closed-form solution.

10.9 Common Mistakes

Side effects inside a comprehension

results = [print(x) for x in range(5)]   # works, but returns [None, None, ...]

Comprehensions are for building collections, not for side effects. Use a regular for loop when you just want to call a function.

Iterating a generator twice

gen = (x ** 2 for x in range(5))
print(list(gen))   # [0, 1, 4, 9, 16]
print(list(gen))   # [] - generator exhausted!

A generator can only be iterated once. Convert to a list if you need to iterate it multiple times.

Overly complex comprehensions

Comprehensions with three or more nested loops, multiple conditions, and complex expressions are hard to read. Prefer a regular loop with comments when logic becomes complex.

10.10 Key Terms

Term	Meaning
list comprehension	[expr for x in iterable if cond] - builds a list
dict comprehension	{key: value for x in iterable} - builds a dict
set comprehension	{expr for x in iterable} - builds a set (no duplicates)
generator expression	(expr for x in iterable) - lazy; yields one value at a time
filter clause	if condition after for; controls which items are included
nested comprehension	Multiple for clauses flattening or combining iterables
lazy evaluation	Computing values on demand rather than all at once
exhausted generator	A generator that has yielded all its values; empty on re-iteration