about

Bits: Generic Python

07/25/2024

Bits Repo Code Bits Repo Docs

Bits: Python Generics

Python does not support generics, it does support type hints that are checked with mypy.py

Synopsis:

This page demonstrates uses of Python Generic type syntax using typing import. The purpose is to quickly acquire some familiarity with syntax and behavior of generic typing.

Python defines a typing module providing support for type hints.
The translator ignores typing. You need to use a static analysis tool like mypy.py to detect type errors.

Demo Notes

All of the languages covered in this demonstration support generics with the exception of JavaScript.

Three of the languages: C++, Rust, and C# provide generics. Python has a typing module that provides type hints, checked with a static analysis tool.

Each generic function or class is a pattern for defining functions and classes of a specific type. Thus a generic class is a pattern for making patterns.

Python and JavaScript, are dynamically typed and already support defining functions and classes for multiple types, e.g., no need for generics.

This demonstration illustrates use of generic functions and classes, which for C++, Rust, and C#, are processed by the compiler.

The examples below show how to use library and user defined types with emphasis on illustrating syntax and basic operations.

1.0 Python Generics and Abstract Base Classes

Python provides type hints to decorate definitions of functions and classes.

from typing import TypeVar, Generic
T = TypeVar('T')

def function(arg: T) -> T:
return arg

Type hints are ignored by the Python translator, but a seperate tool, mypy.py, can be used to perform static type checks when hints are used.

> python3 -m pip install mypy
> mypy program.py

Python classes may also be decorated with type hints:

from typing import TypeVar, Generic
T = TypeVar('T', bound=int | float)

class Hello(Generic[T]):
# code to define class operations

Not only can type hints be statically checked with mypy.py, they also provide maintainers clear messages about intended uses of the annotated code.

Python does not support interfaces, but it does provide standard Abstract Base Classes (ABC).

from collections.abc import Iterable, Sequence

Container	Generic base for containers that supports the in operator to test for membership. Container is a base for Iterable, Sequence, and Mapping.
Iterable	Base for objects that support iteration, e.g., provide __iter()__ method which returns an iterator. The iterator must support the __next()__ method.
Sequence	Objects that support indexing and slicing
Mapping	Base for dictionaries that supports key-based lookup and iteration.

Code authors can also define their own abstract base classes.

2.0 Source Code

Examples below show how to use library and user defined types with emphasis on illustrating syntax and basic operations.

These demonstrations of Python code with generic type hints are partitioned into modules Py_Generic.py, AnalysisGen.py, and PointsGen.py. Discussion of each of these will be presented in separate sections of this page, accessed from links in the left panel.

The page provides demonstrations for:

standard Python library collections
user defined types Hello[T] and Points[T]
generic functions

using one demonstration function for each of these, invoked in execute() function at the end of the program.

2.1 Demonstrate Standard Library Generics

Here you will see demonstrations of List, tuple, and dict standard library collections. This is the first code in which you see Python type hints. These are ignored by the Python translator but enable static type checking with the tool mypy.py.

Alter splitter-bar panel widths by dragging splitter bar or clicking in either panel to expand that panel's width. Default widths are set by setting width style on left panel.

#-- demonstrate std collection types with generic hints

def demo_std_generics():
  Anal.showNote("  demo std generics")
  print()

  Anal.showOp("list[int]")
  l1: list[int] = [1, 2, 3, 2, 1]
  print(type(l1))
  print(l1)
  print()

  Anal.showOp("list[str]")
  l2: list[str] = ["you", "me", "them", "us"]
  print(type(l2))
  print(l2)
  print()

  Anal.showOp("tuple(int, float, str)")
  t: tuple[int, float, str] = (42, -0.5, "forty two")
  print(type(t))
  print(t)
  print()

  Anal.showOp("dict[str, int]")
  d: dict[str, int] = {
      "zero":0, "one":1, "two":2, "three":3
  }
  print(type(d))
  print(d)
  print()


                  
--------------------------------------------------
  demo std generics
--------------------------------------------------

--- list[int] ---
<class 'list'>
[1, 2, 3, 2, 1]


--- list[str] ---
<class 'list'>
['you', 'me', 'them', 'us']




--- tuple(int, float, str) ---
<class 'tuple'>
(42, -0.5, 'forty two')



--- dict[str, int] ---
<class 'dict'>
{'zero': 0, 'one': 1, 'two': 2, 'three': 3}

2.2 Hello

Hello[t] is a simple Python class designed to illustrate type hinting, default function argument values, and properties.

2.2.1 Hello Definition

Things to note:

Python translator does not support generics. It doesn't need them because all varables are dynamically typed, e.g., assume the type of the data they hold.
So you can pass any type as an argument of a given function. That will fail at runtime if an expression using the argument cannot be evaluated.
Python ignores type hints, but those can be used by tool mypy.py to perform static type analyses. With static analysis, based on type hints, you discover errors through simple clear warnings and error messages instead of run-time exceptions, with more verbose error messages.
Python supports default argument values, as shown below.
The language provides properties that encapsulate class data while supporting public member syntax.
Python does not provide access controls. Data is private, by convention, if its name begins with an underscore character, e.g.,
_myprivatedata = 1.5;
Otherwise it is assumed to be public. This convention is tranparent to the Python translator. It won't emit a warning or error if code changes _myprivatedata.

#-- user defined Hello type --#

from typing import TypeVar, Generic
T = TypeVar('T', bound=int | float)

class Hello(Generic[T]):

  # supports constructor notation
  def __init__(self, t: T = 0) -> None:
      self._datum:T = t

  # property supports public member syntax while
  # keeping _datum encapsulated, e.g., not in
  # public interface

  @property
  def value(self):
     return self._datum

  @value.setter
  def value(self, t:T):
      self._datum = t

  @value.getter
  def value(self):
      return self._datum

  # show named value of Point[T] instance.
  # Uses default value for left.

  def show(self, name, left = 0) :
      print(AnalysisGen.indent(left), sep='', end='')
      print(name, ": ", "Hello { ", self._datum, " }", sep='')

Hello[T]:

⇐ Type Hints

⇐ "generic" class using type hint

⇐ "private" _datum

⇐ property definition

2.2.2 Hello Demonstration

Hello[T] is a simple type, just meant to illustrate how user types are defined. It isn't likely to be useful for other applications.

#-- demonstrate user-defined generic types --#

def demo_user_defined_generics():
  Anal.showNote("  demo user-defined generics")
  print()

  Anal.showOp("Hello(42)")
  h1: Hello[int] = Hello(42)
  print(" ", type(h1))
  h1.show("h1", 2)
  print()

  Anal.showOp("Hello() uses default argument")
  h2: Hello[float] = Hello()
  h2.show("h2", 2)
  print()
  Anal.showOp("h2.value = 3.1415927")
  h2.value = 3.1415927
  h2.show("h2", 2)
  print()

  /* points code elided */

}


                  
--------------------------------------------------
  demo user-defined generics
--------------------------------------------------

--- Hello(42) ---
  <class '__main__.Hello'>
  h1: Hello { 42 }



--- Hello() uses default argument ---
  h2: Hello { 0 }


--- h2.value = 3.1415927 ---
  h2: Hello { 3.1415927 }

2.3 Point Class

The Point[T] class represents points with spatial coordinates and a time that represents when something was at that position. Collections of points might represent the trajectory of an aircraft or the state of some evolving chemical process.

2.3.1 Point Definition

Point[T] defines a point in space-time with N spatial coordinates. N is defined by an integer passed to its constructor:

p : Point[float] = Point[float](3)

The time coordinate is set to system time when p is constructed. That value is accessed from its dt property:

timeStr : string = p.dt

At some time after construction, an object's dt can be updated to match current system time with:

p.now()

#----------------------------------------------------------
#   Py_Generic::PointsGen.py
#   Point[T]
#     Point in N dimensional hyperspace with type T
#     coordinates.
#----------------------------------------------------------

import datetime
import time
import AnalysisGen
from typing import TypeVar, Generic

# Generic point class with N coordinates
#   Python doesn't need generics to support types
#   defined at translation time, but they were
#   introduced to enable translation time type
#   error detection.

T = TypeVar('T', bound=int | float)

class Point(Generic[T]):

    # supports constructor notation
    def __init__(self, n: int) -> None:         # p = Point<double>(8)
        self.coors:list[T] = []
        for i in range(n):
            self.coors.append(0)
        self._dt: datetime = datetime.datetime.now()

    def append(self, t: T):
        self.coors.append(t)

    def len(self) -> int:
        return len(self.coors)

    def __len__(self) -> int:                   # len(p)
        return len(self.coors)

    def __getitem__(self, key: int) -> T:       # value = p[1]
        return self.coors[key]

    def __setitem__(self, key: int, val: T):    # p[1] = value
        self.coors[key] = val

    @property
    def dt(self):
        return self._dt

    @dt.getter
    def dt(self):
        return self._dt

    def now(self):
        _dt = datetime.datetime.now()

    # show named value of Point[T] instance
    def show(self, name, left = 2, width = 7) :
        print(AnalysisGen.indent(left), name, ' ', sep='')
        print(AnalysisGen.indent(left), "{", sep='')
        print(AnalysisGen.fold(self.coors, left+3, width), sep='')
        print(AnalysisGen.indent(left + 2), self.dt)
        print(AnalysisGen.indent(left), "}", sep = '')

Point[T]:

Point with n spatial dimensions and
datetime point

⇐ imports

⇐ class definition

⇐ constructor

⇐ indexers

⇐ time property

⇐ display method

2.3.2 Point Demonstration

#-- demonstrate user-defined generic types --#

def demo_user_defined_generics():
  Anal.showNote("  demo user-defined generics")
  print()

  /* Hello code elided */

  Anal.showOp("p1 = Points.Point[float](5)")
  p1: Points.Point = Points.Point[float](5)
  p1[1] = 1.5
  p1[3] = -2.0
  index = p1.len() - 1
  p1[index] = 42
  print(" ", type(p1))
  p1.show("p1", 2)
  print()
}


                  
--------------------------------------------------
  demo user-defined generics
--------------------------------------------------



--- p1 = Points.Point[float](5) ---
  <class 'PointsGen.Point'>
  p1
  {
     0, 1.5, 0, -2.0, 42
     2024-07-24 21:46:01.293971
  }

2.4 Generic Functions

You have already seen definitions of generic methods for the Hello[T] and Point[T] classes. This section illustrates how type hints are used for free standing functions.

2.4.1 Generic Function Definitions

Here, we look at five generic functions that have been used in most of the demonstrations, above:

showType(t:T, nm:str, indnt = 2, suffix = "")
fold(enum: Sequence, left:int, width:int) -> str
showTypeEnum(enum: Sequence, nm: str, left=2, width=7, suffix="")
showTypeShowable(t:T, nm: str, suffix="")
showIdent(t:T, name:str, suffix="")

and four non-generic functions, which use type hints:

indent(n:int) -> str
showNote(text:str, suffix="", n:int=50)
showOp(text: str)

One notable omission in this section: there is no discussion of Python lambdas:

lambda a, b, c, ... : single expression

The single expression limitation makes Python lambdas significantly less useful than lambdas in other languages like C++, Rust, and C#,

#----------------------------------------------------------
#   Py_Generic::AnalysisGen.py
#   - Collection of display and analysis functions
#----------------------------------------------------------

import sys
import collections
from collections.abc import Sequence
from typing import TypeVar

T = TypeVar('T')
# Coll = Union[Iterable, Sized] = 'Coll'

# Python requires definition before use ordering
#  - no link phase to find definitions
import copy
nl = "\n"

# show name, type, value, and size of a Python instance
def showType(t:T, nm:str, indnt:int = 2, suffix:str = "") :
    print(indent(indnt), nm, ' ', type(t), " dynamic", sep="")
    print(indent(indnt), "value: ", t, ', size: ', sys.getsizeof(t), suffix, sep="")

# generate indent string with n spaces
def indent(n:int):
    tmpStr = ""
    for i in range(n):
        tmpStr += ' '
    return tmpStr

# fold indexable into rows of width elements indented by
# left spaces
def fold(enum: Sequence, left:int, width:int) -> str:
    tmpStr = indent(left)
    for i in range(len(enum)):
        tmpStr += str(enum[i]) + ", "
        if(((i + 1) % width) == 0 and i != 0):
            tmpStr += "\n" + indent(left)
    rIndex = tmpStr.rindex(',')
    tmpStr = tmpStr[:rIndex]
    return tmpStr

# show name, type, value, and size of a Python instance
def showTypeEnum(enum:Sequence, nm:str, left:int = 2, width:int = 7, suffix:str = "") :
    # topStr = indent(left) + nm + type(enum) + "dynamic"
    print(indent(left),nm, ' ', type(enum), ' ', "dynamic", sep='')
    print(indent(left), "{", sep='')
    print(fold(enum, left+2, width))
    print(indent(left), "}", sep = '')
    print(indent(left), "size: ", sys.getsizeof(enum), suffix, sep='')

# same as showType except uses class method to show value
def showTypeShowable(t:T, nm:str, suffix:str = ""):
    print(type(t), "dynamic")
    t.show(nm)

# show Python id, unique for each instance
def showIdent(t:T, name:str, suffix:str = "") :
    print(name, '"{}"'.format(t), id(t), suffix)

# show emphasized note
def showNote(text:str, suffix:str = "", n: int = 50) :
    tmpStr = ""
    for i in range(n):
      tmpStr += '-'
    print(tmpStr)
    print(text)
    print(tmpStr, suffix)

# show delineated string to announce a program operation
def showOp(text:str):
    print("--- {} ---".format(text))

Generic Functions:

⇐ imports

⇐ showType displays, in a formatted block, the call name, nm,
Python's inferred type, and the value of the
generic parameter t and its size.

⇐ indent A helper function for the fold function, which
returns a string with n spaces, used for format-
ting output.

⇐ fold A helper function which folds an enumerable
sequence of values into rows

⇐ showTypeEnum Similar to showType functionality except that
its argument is an enumberable sequence instead
of a scalar. Values are displayed in rows.

2.4.2 Generic Function Demonstration


def demo_generic_functions():

  Anal.showNote("  demo generic functions", "\n")

  Anal.showOp("Anal.showType(list[int], name)")
  l1:list[int] = [1, 2, 3, 2, 1, 0, -1, -2, -1, 0]
  Anal.showType(l1, "l1")
  print()

  Anal.showOp("showTypeEnum(list[int], name)")
  Anal.showTypeEnum(l1, "l1", left = 2, width = 7, suffix = "")
  print()


--------------------------------------------------
  demo generic functions
--------------------------------------------------

--- Anal.showType(list[int], name) ---
  l1 <class 'list'> dynamic
  value: [1, 2, 3, 2, 1, 0, -1, -2, -1, 0], size: 136


--- showTypeEnum(list[int], name) ---
  l1 <class 'list'> dynamic
  {
    1, 2, 3, 2, 1, 0, -1,
    -2, -1, 0
  }
  size: 136

2.4.3 Static Analysis

In the block below find the output of checking the AnalysisGen.py module with mypy. It found 1 error in the showTypeShowable(t:T, name:str, suffix:str="") function.

That's because the function body evaluates the expression t.show(nm). But the generic type T is not bound to supply the show method.

C:\github\JimFawcett\Bits\Python\Py_Generic
> mypy AnalysisGen.py 
AnalysisGen.py:55: error: "T" has no attribute "show"  [attr-defined]  
Found 1 error in 1 file (checked 1 source file)
C:\github\JimFawcett\Bits\Python\Py_Generic

To fix that error we would need to define an Abstract Base Class Showable that declares a show(nm:str) method and replace T with Showable. That is left as a non-trivial exercise for the reader.

2.5 Program Structure

Illustrates program structure and control flow by eliding all of the demonstration code details.

#----------------------------------------------------------
# Py_Generic.py
#
# Python type hints and generics
#   - Demonstrates generic and type hint syntax
#   - Shows how to build user-defined generic type
#   - Type hints are ignored by the Python interpreter
#   - To check for type errors use tool mypy, e.g.:
#
#         mypy Py_Generic.py --check-untyped-defs
#
#     mypy requires installation using pip
#----------------------------------------------------------
# Python std types
#  list, tuple, range, dict, set,
import copy
import AnalysisGen
import PointsGen

# short-hand identifiers
Anal = AnalysisGen
Points = PointsGen

#-- demonstrate std collection types with generic hints

def demo_std_generics():
  # code elided, shown above

#-- user defined Hello type --#

from typing import TypeVar, Generic
T = TypeVar('T', bound=int | float)

class Hello(Generic[T]):
  # code elided, shown above

#-- demonstrate user-defined generic types --#

def demo_user_defined_generics():
  # code elided, shown above

def demo_generic_functions():
  # code elided, shown above

#-- demonstration starts here --#

def execute() :
    Anal.showNote("  Demonstrate Python Generics")

    Anal.showNote(
        "  Python introduced generics and type hints\n"\
        "  to check for type errors using mypy, a static\n"\
        "  type checking tool."
    )
    print()

    demo_std_generics()
    demo_user_defined_generics()
    demo_generic_functions()

    print("\nThat's all folks!\n")

execute()

C:\github\JimFawcett\Bits\Python\Py_Generic
> python Py_Generic.py
--------------------------------------------------
  Demonstrate Python Generics
--------------------------------------------------
--------------------------------------------------
  Python introduced generics and type hints
  to check for type errors using mypy, a static
  type checking tool.
--------------------------------------------------

--------------------------------------------------
  demo std generics
--------------------------------------------------

--- list[int] ---
<class 'list'>
[1, 2, 3, 2, 1]

--- list[str] ---
<class 'list'>
['you', 'me', 'them', 'us']

--- tuple(int, float, str) ---
<class 'tuple'>
(42, -0.5, 'forty two')

--- dict[str, int] ---
<class 'dict'>
{'zero': 0, 'one': 1, 'two': 2, 'three': 3}

--------------------------------------------------
  demo user-defined generics
--------------------------------------------------

--- Hello(42) ---
  <class '__main__.Hello'>
  h1: Hello { 42 }

--- Hello() uses default argument ---
  h2: Hello { 0 }

--- h2.value = 3.1415927 ---
  h2: Hello { 3.1415927 }

--- p1 = Points.Point[float](5) ---
  <class 'PointsGen.Point'>
  p1
  {
     0, 1.5, 0, -2.0, 42
  }

--------------------------------------------------
  demo generic functions
--------------------------------------------------

--- Anal.showType(list[int], name) ---
  l1 <class 'list'> dynamic
  value: [1, 2, 3, 2, 1, 0, -1, -2, -1, 0], size: 136

--- showTypeEnum(list[int], name) ---
  l1 <class 'list'> dynamic
  {
    1, 2, 3, 2, 1, 0, -1,
    -2, -1, 0
  }
  size: 136


That's all folks!

C:\github\JimFawcett\Bits\Python\Py_Generic

3.0 Build

C:\github\JimFawcett\Bits\Python\Py_Generic  
> python Py_Generic.py

4.0 VS Code View

The code for this demo is available in github.com/JimFawcett/Bits. If you click on the Code dropdown you can clone the repository of all code for these demos to your local drive. Then, it is easy to bring up any example, in any of the languages, in VS Code.

Here, we do that for Python\Python_Objects.

Figure 1. VS Code IDE - Python Generics

5.0 References

Reference	Description
Python Type Hints	Syntax for type hinting - requires static analysis tool mypy.py to catch type errors.
Python Abstract Base Classes	Specification for building Python's equivalent of Interfaces
Python Tutorial - w3schools	Interactive examples
Python Reference - docs.python.org	Semi-formal syntax reference