Dict values not inferred correctly with .get() calls

[davidbauer99]

The function below results in an unexpected return type error. test.py:6: error: Incompatible return value type (got "Optional[int]", expected "int")

from typing import Dict


def results_in_error(data: Dict[str, int], key: str) -> int:
    if key in data:
        return data.get(key)
    return 0


def works_as_expected(data: Dict[str, int], key: str) -> int:
    val = data.get(key)
    if val:
        return val
    return 0

Replacing if key in data: with if data.get(key): will result in an identical error.

My mypy.ini file is the following:

[mypy]
ignore_missing_imports=True
python_version = 3.6
strict_optional=True
check_untyped_defs=True
disallow_untyped_calls=True
disallow_untyped_defs=True
follow_imports=normal
follow_imports_for_stubs=True

I believe this is a false positive as the value will not be None due to the dict's typing.

[ilevkivskyi]

From typing point of view there is nothing wrong here. For comparison:

def test() -> int:
    if 2 + 2 > 4:
        return 'no way'

will also fail. It is just not possible to special-case all possible patterns. In particular, your original example is actually an anti-pattern. Instead, you should write:

def better_way(data: Dict[str, int], key: str) -> int:
    if key in data:
        return data[key]
    return 0

Or even better:

def even_better_way(data: Dict[str, int], key: str) -> int:
    return data.get(key, 0)

[unformatt]

I don't understand this explanation. Showing a "better" way to write the code doesn't explain why mypy doesn't think that data.get(key) would return an int given that data is typed as Dict[str, int].

[sshishov]

The explanation is simple: when it is executing the line return data.get(key), the signature of this functions is returning Optional[int] because if the key does not exist, it will return None.

It cannot know by this time that the key does exist because otherwise it should store some kind of state machine to know that we have already checked that the key in the dict.

As suggested, we should be more specific and write the code which cannot be interpreted differently. If you expect on this line where you return, that the value DOES EXIST in the dictionary, just return it using data[key] which cannot return None and therefore will not return Optional

[caliraftdude]

I get the difficulty here in trying to check data driven code that might be ambiguous, but the suggestion of using data[key] as an alternative seems more like a suggestion on how to avoid a "mypy" error as opposed to avoiding the program from crashing because of a non-existent key. The exact reason for the get() function is to short circuit a more complex "look for key, validate it exists, then try and retrieve it" paradigm by just returning the value or None.

to work around such a thing, you can:

from typing import Optional
val: Optional[int] = data.get(key)

Which will sort it out and imho, is a better option that writing fragile code to avoid a syntax checking error.

[swirle13]

@caliraftdude because in OP's example, he's already checking if the key exists in the dict data via if key in data:. If the conditional is true, there is no reason to use the .get() method, which is designed to handle if the key exists or not, thus why it returns either None or the value.

So by checking with the if statement and using .get(), he is erroneously mismatching the returned type of the function from int to Optional[int]. He is performing double the checks and is introducing more complex return types, which mypy complains about.

He should do either of these:

def better_way(data: Dict[str, int], key: str) -> int:
    if key in data:
        return data[key]  # key will always exist inside this conditional, thus not unsafe
    return 0

# Or

another_way(data: Dict[str, int], key: str) -> Optional[int]:  # match declared return type for .get()
    return data.get(key)  # will return None if key doesn't exist, otherwise, returns found value

# Or

even_better_way(data: Dict[str, int], key: str) -> int:
    return data.get(key, 0)  # set returned default value of 0 if key is not found to keep return type as int

[hauntsaninja]

Also note that if the default value is expensive to construct, you may want something like data.get(key) or expensive_object() or if your values can be false-y then val if (val := data.get(key)) is not None else expensive_object()

See openai/whisper#370 for a real life case of this bug