about
Bits: Python Iteration
08/01/2024
0
Bits: Python Iteration
iterating through Python standard and user-defined collections
Synopsis:
This page demonstrates uses of Python iterable collections, iteration, and creation of iterators for custom types. The purpose is to quickly acquire some familiarity with syntax and behavior of iteration.- Python standard collections all provide methods that return an iterator.
- Each iterator has a next method that returns the next collection value or, if there are no more, throws an exception.
- User defined types can implement iterators and define methods that return them.
Demo Notes
1.0 Python Iterable Collections and Iterators
Method: coll.__iter()__ -> iterator
and
Functioniter(coll) -> iterator
An iterator, itr, provides:
and
Function
Method: itr.__next()__ -> item
and
Functionnext(itr) -> item
When an iterator has returned all of the items in a collection, another invocation of and
Function
# do something with item
}
Container |
Generic base for containers that supports the |
Iterable |
Base for objects that support iteration, e.g., provide |
Sequence | Objects that support indexing and slicing |
Mapping | Base for dictionaries that supports key-based lookup and iteration. |
2.0 Source Code
- standard Python library collections
- user defined types
Hello[T] ,Stats[T] , andPoints[T] - generic functions
2.1 Iteration over Standard Ranges
# generator doesn't need to load entire collection
#------------------------------------------------
def generator(coll: Iterable[T]) -> Generator:
for item in coll:
yield item
#------------------------------------------------
# executing basic iterations over range
#------------------------------------------------
def execute_basic_iterations():
Anal.showNote(" basic iteration over ranges", "\n")
# iteration using iterator directly
Anal.showOp("extracting iterator from range(0,10)")
itr = range(0,10).__iter__()
print(" ", end='')
while True:
try:
print("{} ".format(itr.__next__()), end='')
except StopIteration:
break
print()
# iteration using python functions iter() and next()
Anal.showOp("extracting iterator from range(0,10) with iter() and next()")
my_itr = iter(range(0,10))
print(" ", end='')
while True:
try:
print("{} ".format(next(my_itr)), end='')
except StopIteration:
break
print()
# idiomatic iteration over range using for loop
Anal.showOp("idiomatic for-in iteration over range(1,6)")
print(" ", end='')
for i in range(1,6):
print("{} ".format(i), end='')
print()
# direct use of generator
Anal.showOp("using iterator returned by generator")
itr = generator(range(1,5))
print(" ", end='')
try:
while True:
print("{} ".format(next(itr)), end='')
except StopIteration:
pass
print()
# iteration using generator
Anal.showOp("idiomatic iteration over range(1,5) using generator")
print(" ", end='')
for item in generator(range(1,5)):
print("{} ".format(item), end='')
print("\n")
--------------------------------------------------
basic iteration over ranges
--------------------------------------------------
--- extracting iterator from range(0,10) ---
0 1 2 3 4 5 6 7 8 9
--- extracting iterator from range(0,10) with iter() and next() ---
0 1 2 3 4 5 6 7 8 9
--- idiomatic for-in iteration over range(1,6) ---
1 2 3 4 5
--- using iterator returned by generator ---
1 2 3 4
--- idiomatic iteration over range(1,5) using generator ---
1 2 3 4
2.2 Iteration over Standard Collections
# demo iterations over standard collections
#------------------------------------------------
# for seq iter returns seq item
def forloop_seq_iteration(coll):
print(type(coll))
itr = coll.__iter__()
print(" {}".format(next(itr)), end='')
for item in itr:
print(", {}".format(item), end='')
print()
# for map iter returns map key
def forloop_map_iteration(coll):
print(type(coll))
itr = coll.__iter__()
key = next(itr)
print(' {', key, ': ', coll[key], '}', end='')
for key in itr:
print(', {', key, ': ', coll[key], '}', end='')
print()
#------------------------------------------------
# executing iteration over std collections
# -----------------------------------------------
def iteration_over_std_collections():
Anal.showNote(" iteration over std collections", "\n")
Anal.showOp("list[float]")
l:list[float] = [1, 2.25, 3.5, 2.75, 1]
forloop_seq_iteration(l)
print()
Anal.showOp("string")
s:str = "a string"
forloop_seq_iteration(s)
print()
Anal.showOp("dictionary[str, int]")
d:dict[str, int] = {
"zero": 0, "one": 1, "two":2
}
d["three"] = 3
forloop_map_iteration(d)
print()
--------------------------------------------------
iteration over std collections
--------------------------------------------------
--- list[float] ---
<class 'list'>
1, 2.25, 3.5, 2.75, 1
--- string ---
<class 'str'>
a, , s, t, r, i, n, g
--- dictionary[str, int] ---
<class 'dict'>
{ zero : 0 }, { one : 1 }, { two : 2 }, { three : 3 }
2.3 Stats[T] Class
2.3.1 Stats[T] Definition
#----------------------------------------------------------
# Py_Generic::Stats.py
# Stats[T]
# Collection of T items with simple statistics.
#----------------------------------------------------------
import AnalysisIter
from typing import TypeVar, Generic, Iterator
# Generic Stats class with list of numeric items
# - Supports iteration
T = TypeVar('T', bound=int | float)
Anal = AnalysisIter
class Stats(Generic[T]):
def __init__(self, l:list[T]) -> None:
self.items = l
def len(self) -> int:
return self.items.len()
def max(self) -> T | None:
if not self.items:
return None
max:T = self.items[0]
for item in self.items:
if max < item:
max = item
return max
def min(self) -> T | None:
if not self.items:
return None
min:T = self.items[0]
for item in self.items:
if min > item:
min = item
return min
def sum(self) -> T:
sum:T = 0
for item in self.items:
sum += item
return sum
def avg(self) -> T | None:
if not self.items:
return None
num = self.sum()
den = len(self.items)
return num/den
def __iter__(self) -> Iterator: # p.iter()
return self.items.__iter__()
2.3.2 Stats[T] Demonstration
#------------------------------------------------
# executing iterations over user-defined types
#------------------------------------------------
def iteration_over_user_defined_types():
Anal.showNote(" iteration over user-defined collections", "\n")
Anal.showOp("Stats[float]")
l:list[float] = [1.0, 2.25, 3.5, 4.75, 5.0]
s:Stats.Stats[float] = Stats.Stats[float](l)
print(l)
print(" max: {}".format(s.max()))
print(" min: {}".format(s.min()))
print(" sum: {}".format(s.sum())) # sum() uses for-in iteration
print(" avg: {}".format(s.avg()))
print()
Anal.showOp("iterating over s:Stats[float]")
print(" ", end='')
for item in s: {
print("{} ".format(item), end='')
}
print("\n")
Anal.showOp("error handling for empty serr: Stats[float]")
serr = Stats.Stats[float](0)
avg = serr.avg()
print(" serr.avg(): ", serr.avg())
print()
/* iteration over points elided */
--------------------------------------------------
iteration over user-defined collections
--------------------------------------------------
--- Stats[float] ---
[1.0, 2.25, 3.5, 4.75, 5.0]
max: 5.0
min: 1.0
sum: 16.5
avg: 3.3
--- iterating over s:Stats[float] ---
1.0 2.25 3.5 4.75 5.0
--- error handling for empty serr: Stats[float] ---
serr.avg(): None
2.4 Point Class
2.4.1 Point Definition
#----------------------------------------------------------
# Py_Generic::PointsGen.py
# Point[T]
# Point in N dimensional hyperspace with type T
# coordinates.
#----------------------------------------------------------
import AnalysisIter
from typing import TypeVar, Generic, Iterator
# Generic point class with N coordinates
# - Supports indexing and iteration
T = TypeVar('T', bound=int | float)
Anal = AnalysisIter
class Point(Generic[T]):
# supports constructor notation
def __init__(self, n: int) -> None: # p = Point<double>(8)
self.coors:list[T] = []
for i in range(n):
self.coors.append(0) # there doesn't seem to be a default
# value for T, e.g., T::default()
def append(self, t: T):
self.coors.append(t)
def len(self) -> int: # p.len()
return len(self.coors)
def __len__(self) -> int: # len(p)
return len(self.coors)
def __getitem__(self, key: int) -> T: # value = p[1]
return self.coors[key]
def __setitem__(self, key: int, val: T): # p[1] = value
self.coors[key] = val
def iter(self) -> Iterator: # p.iter()
return self.coors.__iter__()
# Point does not have to define an iterator.
# It simply uses coors iterator.
# show named value of Point[T] instance
def show(self, name, left = 0, width = 7) :
print(Anal.indent(left), name, ' {', sep='')
print(Anal.fold(self.coors, left+2, width))
print(Anal.indent(left), "}", sep = '')
Point[T]:
Iterable datetime point, and iterator ⇐ imports ⇐ class definition ⇐ constructor ⇐ iterator
2.4.2 Point Demonstration
#------------------------------------------------
# executing iterations over user-defined types
#------------------------------------------------
def iteration_over_user_defined_types():
Anal.showNote(" iteration over user-defined collections", "\n")
# Stats code elided
Anal.showOp("Point[float]")
p:Points.Point[float] = Points.Point[float](0)
p.coors = l # all Python members are public
# alternate initialization
# p.append(1.0)
# p.append(0.5)
# p.append(0.0)
# p.append(-0.5)
# p.append(-1.0)
p.show("p") # show uses fold which uses for-in iteration
print()
Anal.showOp("iterate using p's iterator")
itr = p.iter()
print(" {}".format(next(itr)), end='')
for item in itr : {
print(", {}".format(item), end='')
}
print("\n")
--------------------------------------------------
iteration over user-defined collections
--------------------------------------------------
# Stats code elided
--- Point[float] ---
p {
1.0, 2.25, 3.5, 4.75, 5.0
}
--- iterate using p's iterator ---
1.0, 2.25, 3.5, 4.75, 5.0
2.5 Generic Functions
2.5.1 Generic Function Definitions
#-----------------------------------------------------
# demo formatting functions over generic collections
#-----------------------------------------------------
# forloop extracts iterator from enumerable
#------------------------------------------------
def forloopFormatted(enum: Sequence, nm:str):
print(" ", nm, type(enum))
print(" ", end='')
first:bool = True
for item in enum:
if first:
print(" {}".format(item), end='')
first = False
else:
print(", {}".format(item), end='')
print()
# forloopFormattedFolded uses AnalysisIter function fold
#------------------------------------------------
def forloopFormattedFolded(enum:Sequence, nm:str, left, width):
print(nm)
s = Anal.fold(enum, left, width) # uses for loop
print(s)
# forloopAssocFolded uses fold function implemented for
# associative containers
#------------------------------------------------
def forloopAssocFolded(enum:Mapping, nm:str, left, width):
print(nm)
s = Anal.foldAssoc(enum, left, width) # uses for loop
print(s)
Generic Functions:
⇐⇐
⇐
2.5.2 Generic Function Demonstration
#------------------------------------------------
# demo executing formatted iteration over
# std collections
#------------------------------------------------
def iteration_using_formatting_functions():
Anal.showNote(" iteration_using_formatting_functions")
print()
Anal.showOp("forloopFormatted(list[int])")
l:list[int] = [1, 2, 3, 2, 1]
forloopFormatted(l, "l")
print()
Anal.showOp("forloopFormatted(str)")
s:str = "a string"
forloopFormatted(s, "str")
print()
Anal.showOp("Anal.showTypeEnum(range(1,6), ...)")
Anal.showTypeEnum(range(1,6), "range(1,6)", 2, 7)
print()
Anal.showOp("forloopFormattedFolded(list)")
l:list[float] = [1.0, 2.5, -3.5, 2.0, 1.5, 0.5]
forloopFormattedFolded(l, " list[float]", 4, 4)
print()
Anal.showOp("forloopAssocFolded(dict)")
d:dict[int, str] = {
1:"one", 2:"two", 3:"three", 4:"four", 5:"five"
}
forloopAssocFolded(d, " dictionary[int, str]", 4, 4)
print()
#------------------------------------------------
# executing formatting iterations over user-defined
# types
# -----------------------------------------------
def iteration_formatted_user_defined_types():
Anal.showNote(" formatted iteration over user-defined type", "\n")
Anal.showOp("p.show('p', 0, 7)")
p = Points.Point[float](10)
p[1] = 1
p[3] = 3.5
p[9] = 42
p.show("p", 2, 7)
print()
Anal.showOp(
"forloopFormattedFolded(p, 'p : Points...', 4, 7)"
)
forloopFormattedFolded(p, " p : Points.Point[float](10)", 4, 7)
--------------------------------------------------
iteration_using_formatting_functions
--------------------------------------------------
--- forloopFormatted(list[int]) ---
l <class 'list'>
1, 2, 3, 2, 1
--- forloopFormatted(str) ---
str <class 'str'>
a, , s, t, r, i, n, g
--- Anal.showTypeEnum(range(1,6), ...) ---
range(1,6) <class 'range'> dynamic
{
1, 2, 3, 4, 5
}
size: 48
--- forloopFormattedFolded(list) ---
list[float]
1.0, 2.5, -3.5, 2.0,
1.5, 0.5
--- forloopAssocFolded(dict) ---
dictionary[int, str]
{ 1 : one }, { 2 : two }, { 3 : three }, { 4 : four },
{ 5 : five }
--------------------------------------------------
formatted iteration over user-defined type
--------------------------------------------------
--- p.show('p', 0, 7) ---
p {
0, 1, 0, 3.5, 0, 0, 0,
0, 0, 42
}
--- forloopFormattedFolded(p, 'p : Points...', 4, 7) ---
p : Points.Point[float](10)
0, 1, 0, 3.5, 0, 0, 0,
0, 0, 42
2.5 Analysis and Display Functions
fold(enum: Sequence, left:int, width:int) -> str
showTypeEnum(enum: Sequence, nm: str, left=2, width=7, suffix="")
showTypeShowable(t:T, nm: str, suffix="")
showIdent(t:T, name:str, suffix="")
showNote(text:str, suffix="", n:int=50)
showOp(text: str)
2.5.1 Analysis Definitions
#----------------------------------------------------------
# Py_Generic::AnalysisGen.py
# - Collection of display and analysis functions
#----------------------------------------------------------
import sys
from typing import TypeVar
from collections.abc import Sequence, Mapping
T = TypeVar('T')
nl = "\n"
# show name, type, value, and size of a Python instance
def showType(t:T, nm:str, suffix = "") :
print(nm, type(t), "dynamic")
print("value: ", t, ', size: ', sys.getsizeof(t), suffix)
# generate indent string with n spaces
def indent(n:int):
tmpStr = ""
for i in range(n):
tmpStr += ' '
return tmpStr
# fold indexable into rows of width elements indented by
# left spaces
def fold(enum: Sequence, left:int, width:int) -> str:
tmpStr = indent(left)
for i in range(len(enum)):
tmpStr += str(enum[i]) + ", "
if(((i + 1) % width) == 0 and i != 0):
tmpStr += "\n" + indent(left)
rIndex = tmpStr.rindex(',')
tmpStr = tmpStr[:rIndex]
return tmpStr
# fold indexable into rows of width elements indented by
# left spaces
def foldAssoc(enum: Mapping, left:int, width:int) -> str:
keys = enum.keys()
tmpStr = indent(left)
for i in keys:
tmpStr += "{ " + str(i) + " : " + str(enum[i]) + " }" + ", "
if((i % width) == 0 and i != 0):
tmpStr += "\n" + indent(left)
rIndex = tmpStr.rindex(',')
tmpStr = tmpStr[:rIndex]
return tmpStr
# show name, type, value, and size of a Python instance
def showTypeEnum(enum, nm, left = 2, width = 7, suffix = "") :
# topStr = indent(left) + nm + type(enum) + "dynamic"
print(indent(left),nm, ' ', type(enum), ' ', "dynamic", sep='')
print(indent(left), "{", sep='')
print(fold(enum, left+2, width))
print(indent(left), "}", sep = '')
print(indent(left), "size: ", sys.getsizeof(enum), suffix, sep='')
# same as showType except uses class method to show value
def showTypeShowable(t, nm, suffix = ""):
print(type(t), "dynamic")
t.show(nm)
# show Python id, unique for each instance
def showIdent(t, name, suffix = "") :
print(name, '"{}"'.format(t), id(t), suffix)
# show emphasized note
def showNote(text, suffix = "", n: int = 50) :
tmpStr = ""
for i in range(n):
tmpStr += '-'
print(tmpStr)
print(text)
print(tmpStr, suffix)
# show delineated string to announce a program operation
def showOp(text):
print("--- {} ---".format(text))
Analysis and Display Functions:
⇐⇐
⇐
⇐
⇐
Note: Python does not have interfaces to
provide constraints, so may fail at runtime.
2.6 Program Structure
Bits_IterPython
#----------------------------------------------------------
# Py_Iter.py
#
# Python type hints and generics
# - Demonstrates generic and type hint syntax
# - Shows how to build user-defined generic type
# - Type hints are ignored by the Python interpreter
# - To check for type errors use tool mypy, e.g.:
#
# mypy Py_Generic.py --check-untyped-defs
#
# mypy requires installation using pip
#----------------------------------------------------------
# import copy
# import collections
from typing import TypeVar, Iterable, Tuple, Generator
from collections.abc import Sequence, Mapping
import AnalysisIter
import PointsIter
import Stats
T = TypeVar('T')
# short-hand identifiers
Anal = AnalysisIter
Points = PointsIter
# generator doesn't need to load entire collection
#------------------------------------------------
def generator(coll: Iterable[T]) -> Generator:
for item in coll:
yield item
#------------------------------------------------
# executing basic iterations over range
#------------------------------------------------
def execute_basic_iterations():
# code elided
#------------------------------------------------
# executing iteration over std collections
# -----------------------------------------------
def iteration_over_std_collections():
# code elided
#------------------------------------------------
# executing iterations over user-defined types
#------------------------------------------------
def iteration_over_user_defined_types():
# code elided
#------------------------------------------------
# demo executing formatted iteration over
# std collections
#------------------------------------------------
def iteration_using_formatting_functions():
# code elided
#------------------------------------------------
# executing formatting iterations over user-defined
# types
# -----------------------------------------------
def iteration_formatted_user_defined_types():
# code elided
#------------------------------------------------
# function implementing all demonstrations
#------------------------------------------------
def execute() :
print()
Anal.showNote(" Demonstrate Python Iterators", "\n")
execute_basic_iterations()
iteration_over_std_collections()
iteration_over_user_defined_types()
iteration_using_formatting_functions()
iteration_formatted_user_defined_types()
print("\n\nThat's all folks!\n")
execute()
> py Py_Iter.py
--------------------------------------------------
Demonstrate Python Iterators
--------------------------------------------------
--------------------------------------------------
basic iteration over ranges
--------------------------------------------------
--- extracting iterator from range(0,10) ---
0 1 2 3 4 5 6 7 8 9
--- extracting iterator from range(0,10) with iter() and next() ---
0 1 2 3 4 5 6 7 8 9
--- idiomatic for-in iteration over range(1,6) ---
1 2 3 4 5
--- using iterator returned by generator ---
1 2 3 4
--- idiomatic iteration over range(1,5) using generator ---
1 2 3 4
--------------------------------------------------
iteration over std collections
--------------------------------------------------
--- list[float] ---
<class 'list'>
1, 2.25, 3.5, 2.75, 1
--- string ---
<class 'str'>
a, , s, t, r, i, n, g
--- dictionary[str, int] ---
<class 'dict'>
{ zero : 0 }, { one : 1 }, { two : 2 }, { three : 3 }
--------------------------------------------------
iteration over user-defined collections
--------------------------------------------------
--- Stats[float] ---
[1.0, 2.25, 3.5, 4.75, 5.0]
max: 5.0
min: 1.0
sum: 16.5
avg: 3.3
--- iterating over s:Stats[float] ---
1.0 2.25 3.5 4.75 5.0
--- error handling for empty serr: Stats[float] ---
serr.avg(): None
--- Point[float] ---
p {
1.0, 2.25, 3.5, 4.75, 5.0
}
--- iterate using p's iterator ---
1.0, 2.25, 3.5, 4.75, 5.0
--------------------------------------------------
iteration_using_formatting_functions
--------------------------------------------------
--- forloopFormatted(list[int]) ---
l <class 'list'>
1, 2, 3, 2, 1
--- forloopFormatted(str) ---
str <class 'str'>
a, , s, t, r, i, n, g
--- Anal.showTypeEnum(range(1,6), ...) ---
range(1,6) <class 'range'> dynamic
{
1, 2, 3, 4, 5
}
size: 48
--- forloopFormattedFolded(list) ---
list[float]
1.0, 2.5, -3.5, 2.0,
1.5, 0.5
--- forloopAssocFolded(dict) ---
dictionary[int, str]
{ 1 : one }, { 2 : two }, { 3 : three }, { 4 : four },
{ 5 : five }
--------------------------------------------------
formatted iteration over user-defined type
--------------------------------------------------
--- p.show('p', 0, 7) ---
p {
0, 1, 0, 3.5, 0, 0, 0,
0, 0, 42
}
--- forloopFormattedFolded(p, 'p : Points...', 4, 7) ---
p : Points.Point[float](10)
0, 1, 0, 3.5, 0, 0, 0,
0, 0, 42
That's all folks!
C:\github\JimFawcett\Bits\Python\Py_Iter
3.0 Build
C:\github\JimFawcett\Bits\Python\Py_Iter
> py Py_Iter.py
4.0 VS Code View
5.0 References
Reference | Description |
---|---|
python-iterators: realpython.com | Detailed, but relatively simple tutorial on Python iterators. |
Python Type Hints | Syntax for type hinting - requires static analysis tool mypy.py to catch type errors. |
Python Abstract Base Classes | Specification for building Python's equivalent of Interfaces |
Python Tutorial - w3schools | Interactive examples |
Python Reference - docs.python.org | Semi-formal syntax reference |
Stats[T]:
Compute statistics on list values ⇐ imports ⇐ class definition ⇐ constructor ⇐ iterator