Iterators - Python Iterators and itertools

S6.0 What This Teaches

This tutorial covers Python's iterator protocol and the itertools module:

The iterator protocol: __iter__ and __next__
Generator functions with yield
Built-in iterator tools: map, filter, zip, enumerate
itertools: chain, islice, groupby, product
Lazy evaluation and memory efficiency

S6.1 The Iterator Protocol

Any object with __iter__ and __next__ is an iterator. for loops call these automatically. You can also call them manually:

lst = [10, 20, 30]
it = iter(lst)     # calls lst.__iter__()
print(next(it))    # 10
print(next(it))    # 20
print(next(it))    # 30
# next(it)         # StopIteration - iterator exhausted

# Any iterable works in for
for x in iter(lst):
    print(x)

Iterators are lazy: they produce one value at a time without building the entire sequence in memory. This is why range(1_000_000) is fast and memory-efficient.

S6.2 Generator Functions

A function containing yield is a generator function. Calling it returns a generator object - an iterator that produces values on demand:

def count_up(start: int, stop: int):
    n = start
    while n < stop:
        yield n
        n += 1

for n in count_up(1, 6):
    print(n, end=" ")   # 1 2 3 4 5

def fibonacci():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

# Take first 10 Fibonacci numbers
import itertools
fibs = list(itertools.islice(fibonacci(), 10))
print(fibs)   # [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]

S6.3 Built-in Iterator Tools

numbers = [1, 2, 3, 4, 5, 6]

# map - apply function to each element (lazy)
squares = map(lambda x: x**2, numbers)

# filter - keep elements satisfying predicate (lazy)
evens = filter(lambda x: x % 2 == 0, numbers)

# zip - pair elements from multiple iterables
names  = ["Alice", "Bob", "Carol"]
scores = [92, 78, 95]
paired = list(zip(names, scores))   # [('Alice', 92), ('Bob', 78), ('Carol', 95)]

# enumerate - add index to any iterable
for i, name in enumerate(names, start=1):
    print(f"{i}. {name}")

# sorted, min, max work on any iterable
print(sorted(map(lambda x: x**2, numbers)))   # [1, 4, 9, 16, 25, 36]

S6.4 itertools

import itertools

# chain - concatenate iterables
combined = list(itertools.chain([1, 2], [3, 4], [5]))   # [1, 2, 3, 4, 5]

# islice - lazy slice of any iterable
first5 = list(itertools.islice(range(100), 5))   # [0, 1, 2, 3, 4]

# groupby - group consecutive equal keys (sort first!)
data = [("A", 1), ("A", 2), ("B", 3), ("B", 4), ("A", 5)]
data.sort(key=lambda x: x[0])
for key, group in itertools.groupby(data, key=lambda x: x[0]):
    print(key, list(group))

# product - Cartesian product
for r, c in itertools.product("ABC", [1, 2]):
    print(r, c, end=" | ")

# accumulate - running totals
import operator
running = list(itertools.accumulate([1, 2, 3, 4, 5]))   # [1, 3, 6, 10, 15]

S6.5 Custom Iterators

class Countdown:
    def __init__(self, start: int):
        self.current = start

    def __iter__(self):
        return self

    def __next__(self) -> int:
        if self.current <= 0:
            raise StopIteration
        val = self.current
        self.current -= 1
        return val

for n in Countdown(5):
    print(n, end=" ")   # 5 4 3 2 1

In practice, generator functions are simpler than implementing the full iterator protocol class. Reserve classes for stateful iterators that need methods beyond iteration.

S6.6 Example - All Together

# Iterators - pipeline processing with generators and itertools.

import itertools

def read_numbers():
    """Simulate reading a stream of numbers."""
    yield from [3, 1, 4, 1, 5, 9, 2, 6, 5, 3, 5]

def deduplicate(seq):
    seen = set()
    for item in seq:
        if item not in seen:
            seen.add(item)
            yield item

def running_average(seq):
    total = 0
    count = 0
    for item in seq:
        total += item
        count += 1
        yield total / count

# Build a lazy pipeline
pipeline = running_average(deduplicate(read_numbers()))
for avg in itertools.islice(pipeline, 5):
    print(f"{avg:.2f}", end=" ")

Expected output:

3.00 2.00 2.67 2.50 3.40

S6.7 Exercise

Exercise

Write a generator powers(base, n) that yields base⁰ through baseⁿ. Consume it with a comprehension to build a list.
Use itertools.groupby on a sorted list of words to group them by their first letter. Print each group.
Build a lazy pipeline that reads integers from range(1, 101), keeps only those divisible by 3 or 5, squares them, and computes the sum using sum() without materializing a list.

S6.8 Common Mistakes

Consuming an iterator twice

it = iter([1, 2, 3])
print(list(it))   # [1, 2, 3]
print(list(it))   # [] - iterator is exhausted!

Convert to a list if you need to iterate multiple times. Iterators are single-pass.

itertools.groupby requires sorted input

data = [1, 2, 1, 2]
for k, g in itertools.groupby(data):
    print(k, list(g))
# 1 [1]  2 [2]  1 [1]  2 [2]  - NOT grouped!
# Sort first: for k, g in itertools.groupby(sorted(data))

Using map/filter where a comprehension is clearer

[x**2 for x in nums if x % 2 == 0] is usually more readable than map(lambda x: x**2, filter(lambda x: x%2==0, nums)). Use map/filter when passing a named function rather than a lambda.

S6.9 Key Terms

Term	Meaning
iterator	Object with __iter__ and __next__; produces one value at a time
iterable	Object with __iter__ that returns an iterator; lists, strings, dicts, etc.
yield	Suspends a generator function and produces the yielded value
generator	Function with yield; returns an iterator that computes values lazily
StopIteration	Exception signaling that an iterator has no more values
itertools.chain	Concatenates multiple iterables without copying
itertools.islice	Takes the first n elements of any iterable lazily
itertools.groupby	Groups consecutive equal-key elements; input must be sorted by key