An attempt to add generic support for TypedDict

Mission complete 🎉. Now available with Python 3.11. Backported to Python <= 3.10 through typing_extensions by @AlexWaygood.

Goal

To extend typing.TypedDict PEP 589 with generic value support. At the moment TypedDict is a special type that is not paramterizable. We need to add runtime support for it through stdlib and provide a type consistency specification for type checkers.

from typing import TypedDict, TypeVar, Generic
T = TypeVar("T")

class TD(Generic[T], TypedDict):
    f1: List[T]

def test_td(aa: TD[T], bb: TD[T]):
    return aa["f1"], bb["f1"]

td1 = test_td({"f1": ["foo"]}, {"f1": ["bar"]})
reveal_type(td1) # info: Type of "td1" is "tuple[List[str], List[str]]"

Timeline

Jan 19 2021:

• Opened PR in pyright
• Requested by maintainers to get approval through the typing-sig or PEP drafting process
• Send out email to typing-sig

Jan 24 2021:

• Advice in typing-sig mailing list with a sketch on how to move forward with a PEP

...

Aug 1 2021:

• PEP ready for review
• Prototype patch for typing.py
• Prototype patch for typing_extensions.py

Aug 4 2021:

• Send out email to typing-sig for comments

Aug 6 2021:

• Foward references are proving to be an issue with the current alternative syntax spec, because _collect_type_vars can't look inside ForwardRef.
• Considering that annotations are not meant for run time, the two previous patches which try to resolve generic params may be too complicated.
• Create new patch for typing.py

Aug 7 2021:

• Create issue in bugs.python.org and cpython pull request

Sep 1st week 2021:

• Including @uriyyo's suggestions in simplifying the implementaion and test cases

• Including @sobolevn suggested test case which found a bug

• Addressing @Fidget-Spinner's concerns

• Pending issue on whether to proxy the dunders like __total__ through the GenericAlias

  class A(TypedDict, Generic[T]):
    foo: T
  
  assert(A[str].__required_keys__  == frozenset(['foo']) # Raises Attribute error
  assert(A[str].__origin__.__required_keys__  == frozenset(['foo']) # Works

  Current personal decision to not proxy dunder attributes because:

  class A(Generic[T]):
    a: T
  
  print(A.__annotations__) # {'a': ~T}
  print(A[str].__annotations__) # raises AttributeError
  print(A[str].__origin__.__annotations__) # {'a': ~T}

• Found a bug in the upstream implementation. Non generic TypedDicts are subscriptable. Not sure if this should be fixed in this PR by overriding __getitem__

  class G(TypedDict):
    g: str
  
  print(G[int]) # G is subscriptable although not a Generic

8 Sep 2021:

• Email sent to typing-sig mailing list: Viability of letting TypedDict classes inherit from Generic without a PEP change

28 April 2022:

• PyCon 2022 Typing Summit "The future of TypedDict" by David Foster breathed new life into this feature.

29 April 2022:

• @JelleZijlstra resumed discussion in the PR and rest of the work that were needed for completion resumed.

3 May 2022

• PR27663 is merged and CPython runtime support for Generic TypedDict will be available from Python 3.11.

Implementations

• Generic TypedDict in typing.py
  • Simple patch that allows inherit from Generic
    • Integrate to test suite
  • Patch with resolving annotation
    • Integrate to test suite
• Generic TypedDict runtime in typing_extensions
  • Patch typing_extensions.py
  • Integrate to test suite
• Generic TypedDict in one of the type checkers (pyright)
  • Patch to Pyright (PR).
  • Rebase the PR on to a current version
  • Support for declaring with TypedDict constructor notation
  • Adding proper test cases to the automated test suite