TypedDict Nested Dictionaries?

[riley-martine]

Is there a way to use TypedDict for more complicated nested structures? For example, if I pass around a dictionary with certain defined keys and values, some of which are dictionaries themselves, can I validate those?

• Are you reporting a bug, or opening a feature request?
  Asking a question/maybe feature request

• Please insert below the code you are checking with mypy,
  or a mock-up repro if the source is private. We would appreciate
  if you try to simplify your case to a minimal repro.

from mypy_extensions import TypedDict

Data = TypedDict("Data", {"i": {"j": int}})

def access(data: Data) -> None:
    data["i"]["j"] += 1

mydata = {"i": {"j": 5}}  # type: Data
access(mydata)

• What is the actual behavior/output?

typeddict_practice.py:3: error: Invalid field type
typeddict_practice.py:7: error: TypedDict "TypedDict" has no key 'i'
typeddict_practice.py:10: error: Extra key 'i' for TypedDict "TypedDict"

• What is the behavior/output you expect?
  No output

• What are the versions of mypy and Python you are using?
  mypy 0.600
  Python 3.6.5

• What are the mypy flags you are using? (For example --strict-optional)
  None

I can use something like Data = TypedDict("Data", {"i": Dict[str, int]}), but that doesn't validate keys past the first level.

I can also do

Intermediate = TypedDict("Intermediate", {"j": int})
Data = TypedDict("Data", {"i": Intermediate})

but this is cumbersome, especially when there are a ton of fields. As #4299 notes, you can't nest TypedDicts either. Is there a better way?

[ilevkivskyi]

But why don't you like sequential definitions? As PEP 20 says flat is better than nested. Also it would make sense to give meaningful names to sub-dictionaries (after all something gave you the idea to use a nested dict instead of a flat one), you can potentially re-use some of them. For example:

class UserData(TypedDict):
    name: str
    int: id

class Request(TypedDict):
    request: bytes
    user: UserData

class Response(TypedDict):
    header: bytes
    load: List[bytes]
    user: UserData

I could imagine sometimes you may want a one-off TypedDict, but IMO such situations are rare so this is a low priority.

[riley-martine]

But why don't you like sequential definitions?

I started implementing with sequential definitions, and it works better than I expected. I was able to reuse some of the inner ones, and it was quite convenient. Thank you for your suggestion!

As PEP 20 says flat is better than nested.

The situation I'm dealing with is server config data returned from an API. It can be highly nested, and takes a lot of named sub-TypedDicts to work. I like the peace of mind knowing that any field accessed is guaranteed to be there and be the right type, that haven't skipped or mistyped an index, and if I create/modify the same type of structure in different places in my code, it is guaranteed to have all of the fields it should.

For example, there is one place in the code where if the server can't be reached, a stub config is returned. Adding the TypedDict code let me know this stub was missing a few fields that could have been accessed later on. Furthermore, adding Optional[]s allowed me to fix places where there wasn't code to handle the absence of a field. (e.g. calling outerdict["key"]["data"].keys() when outerdict["key"]["data"] was type Optional[Dict[str, float]])

such situations are rare

I'm not sure how rare this is, so I will defer to your judgement. Thank you for considering this idea!

[ABCDeath]

I have a question about nested TypedDict too.

from typing import TypedDict

class Foo(TypedDict):
    foo: str

class Bar(TypedDict):
    bar: Foo
    
data: Bar = {
    'bar': {'foo': '1'}
}

print({k: v.get('foo') for k, v in data.items()})

This code works and prints {'bar': '1'} to stdout. But mypy shows an error: error: "object" has no attribute "get"
https://mypy-play.net/?mypy=latest&python=3.8&gist=b74af38f157707d1b5e5558d3ecea9bc
Why does it define the type of a value, returned by .items(), as object? How could I fix this error?

[KeynesYouDigIt]

@ABCDeath
You could try an overrwite to .items? its painful but its the only thought I had.

[FreCap]

To add to @riley-martine example of configs, I often find myself creating message-passing systems (e.g. celery, kafka etc..).

All of these need well defined message schemas that will be passed around producers and consumers.
All of the messages are highly isolated one with the other.
This could probably help generating avro schemas type checkers.

In those cases TypeScript-like definitions make static checkers's life (and developers') somewhat easier.

e.g.

export interface EmailFetcherEvent {
    userEmail: string,
    pageToken?: string | null,
    updateHistoryId?: boolean | false,
    parameters: Array<{
       value: number,
        name: string,
        active: boolean
    }>
}

Thank you for considering this idea!

MyPy Playground:
https://mypy-play.net/?mypy=latest&python=3.9&gist=29faabf0dd252d7772d2d9e01e8ce5a4

[tuukkamustonen]

I guess sequential definitions make sense, when you want to / want to re-use the inner types in multiple places. Like:

class Inner(TypedDict):
    ...

class Outer1(TypedDict):
    field1: str
    inner: Inner

class Outer2(TypedDict):
    field2: str
    inner: Inner

For me, at least, this is rarely the case. Having to do flat/sequential definitions just adds unnecessary verbosity and boilerplate. If an inner type has no use outside of its context, an "in-place" placement (like in TypeScript) would be much nicer.

(I love how) Python's syntax is often short and straight-to-the-point, but this one introduces unnecessary bloat.

[munael]

Still very much a needed QoL. Something like:

Foo = TypedDict('Foo',
{
	'x': {
		'y': [str],
		'z': { 'foo': Path },
	},
})

Notice how literal lists and dicts are used as part of constructing the schema of the TypedDict, alongside otherwise normal types (str, Path, ...).

would be great. Not sure how to combine union type syntax with variable-as-types syntax (e.g. 'y': str | [str]).

[gravesee]

Is it feasible to implement this feature with a mypy plugin? I'm not familiar enough with the model to know whether this information is even analyzed.

[betafcc]

FYI seems to work in pyright microsoft/pyright#4016

from typing import TypedDict

Geometry = TypedDict(
    "Geometry",
    {
        "position": TypedDict("Position", {"x": int, "y": int, "z": int}),
        "dimensions": TypedDict("Dimensions", {"width": int, "height": int}),
    },
)


def area(geometry: Geometry):
    return geometry["dimensions"]["width"] * geometry["dimensions"]["height"]


r = area({"position": {"x": 0, "y": 0, "z": 0}, "dimensions": {"width": 10, "height": 20}})

reveal_type(r) # Revealed type is 'builtins.int'

[erictraut]

@betafcc, pyright should not allow this. It's a false negative, which I will fix. Call expressions are forbidden in type annotations, so this should not be allowed.