Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TypedDict update() does not accept TypedDict with compatible subset keys #6462

Closed
yuyangguo42 opened this issue Feb 21, 2019 · 6 comments · Fixed by #15425
Closed

TypedDict update() does not accept TypedDict with compatible subset keys #6462

yuyangguo42 opened this issue Feb 21, 2019 · 6 comments · Fixed by #15425
Labels
bug mypy got something wrong false-positive mypy gave an error on correct code needs discussion priority-1-normal topic-typed-dict

Comments

@yuyangguo42
Copy link

from mypy_extensions import TypedDict

A = TypedDict(
    'A', {
        'foo': int,
        'bar': int,
    }
)

B = TypedDict(
    'B', {
        'foo': int,
    }
)

a = A({'foo': 1, 'bar': 2})
b = B({'foo': 2})
a.update({'foo': 2})
a.update(b)

Expect this to pass type checking but got this error:

test.py:19: error: Argument 1 to "update" of "TypedDict" has incompatible type "B"; expected "TypedDict({'foo'?: int, 'bar'?: int})"
@JukkaL
Copy link
Collaborator

JukkaL commented Feb 22, 2019

This is a pretty subtle issue. The update call is arguably not type safe. In a more complex program, the argument could be a value of a subtype of B that has the key 'bar' with a list value, for example. This would break type safety, as a['bar'] would be a list instead of an integer.

However, the same reasoning could be applied to update in general, and it turns out that the way update is treated is generally unsafe. There could be two values of different subtypes of A with incompatible values for some key, and if we are looking at both through type A, first.update(second) could again break type safety.

Not sure what should be done. If we want to be safe, we could require that the argument to update is a dict literal. If we want to be flexible, we should probably allow the original example above.

An explicit cast to A is perhaps the most reasonable workaround: a.update(cast(A, b))

@sanjioh
Copy link

sanjioh commented Oct 16, 2019

Hi,

I get the same mypy error with the following code:

from mypy_extensions import TypedDict


class Movie(TypedDict):
    name: str
    year: int


m1 = Movie(name='Blade Runner', year=1982)
m2 = Movie(name='Star Wars', year=1977)
m1.update(m2)
error: Argument 1 to "update" of "TypedDict" has incompatible type "Movie"; expected "TypedDict({'name'?: str, 'year'?: int})"

Is this something related to the described issue?

Thanks!

@ilevkivskyi
Copy link
Member

Is this something related to the described issue?

Yes, I think this is an aspect of the same problem. In this case however it looks to me as a bug, mypy is overly strict in your case, since we know dict.update() never modifies its argument.

@ilevkivskyi ilevkivskyi added bug mypy got something wrong false-positive mypy gave an error on correct code priority-1-normal labels Oct 16, 2019
@mr-c
Copy link
Contributor

mr-c commented May 20, 2020

Hello, the bug in #6462 (comment) just bit me and is blocking my adoption of TypedDict.

Cheers,

@kratsg
Copy link

kratsg commented Jul 1, 2022

I've tried two ways of doing this. One is to iterate over the keys and assign (doesn't work), but this version seems to be the only thing that mypy is happy with

from mypy_extensions import TypedDict


class Movie(TypedDict):
    name: str
    year: int


m1 = Movie(name='Blade Runner', year=1982)
m2 = Movie(name='Star Wars', year=1977)

m1['name'] = m2['name']
m1['year'] = m2['year']

@MiguelGuthridge
Copy link

It doesn't even let you update a TypedDict with itself. Is it even possible to call the function at all?

class TestType(TypedDict):
    prop: int


def doThing(foo: TestType):
    foo.update(foo)  # (parameter) foo: TestType
                     # Argument 1 to "update" of "TypedDict" has incompatible type "TestType"; expected "TypedDict({'prop'?: int})"

xDaile added a commit to xDaile/iib that referenced this issue Oct 10, 2022
Method for updating the dictionary is ignored, as the update method of typedict have some problems (python/mypy#6462).
xDaile added a commit to xDaile/iib that referenced this issue Oct 11, 2022
Method for updating the dictionary is ignored, as the update method of typedict have some problems (python/mypy#6462).
xDaile added a commit to xDaile/iib that referenced this issue Oct 14, 2022
Method for updating the dictionary is ignored, as the update method of typedict have some problems (python/mypy#6462).
xDaile added a commit to xDaile/iib that referenced this issue Oct 18, 2022
Method for updating the dictionary is ignored, as the update method of typedict have some problems (python/mypy#6462).
ilevkivskyi added a commit that referenced this issue Jun 26, 2023
Fixes #9408
Fixes #4122
Fixes #6462
Supersedes #13353

This PR enables two similar technically unsafe behaviors for TypedDicts,
as @JukkaL explained in
#6462 (comment)
allowing an "incomplete" TypedDict as an argument to `.update()` is
technically unsafe (and a similar argument applies to `**` syntax in
TypedDict literals). These are however very common patterns (judging
from number of duplicates to above issues), so I think we should support
them. Here is what I propose:
* Always support cases that are safe (like passing the type itself to
`update`)
* Allow popular but technically unsafe cases _by default_
* Have a new flag (as part of `--strict`) to fall back to current
behavior

Note that unfortunately we can't use just a custom new error code, since
we need to conditionally tweak some types in a plugin. Btw there are
couple TODOs I add here:
* First is for unsafe behavior for repeated TypedDict keys. This is not
new, I just noticed it when working on this
* Second is for tricky corner case involving multiple `**` items where
we may have false-negatives in strict mode.

Note that I don't test all the possible combinations here (since the
phase space is huge), but I think I am testing all main ingredients (and
I will be glad to add more if needed):
* All syntax variants for TypedDicts creation are handled
* Various shadowing/overrides scenarios
* Required vs non-required keys handling
* Union types (both as item and target types)
* Inference for generic TypedDicts
* New strictness flag

More than half of the tests I took from the original PR #13353
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug mypy got something wrong false-positive mypy gave an error on correct code needs discussion priority-1-normal topic-typed-dict
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants