mypy doesn't distinguish between a bool and an int?

[PedanticHacker]

I have encountered a strange mypy error.

My function is defined like this:

def load_settings(table_name: str, key_name: str) -> bool | int:
    """Loads a setting value by the given table_name and key_name."""
    with open(constants.SETTINGS_FILE_PATH, "rb") as settings_file:
        settings = tomllib.load(settings_file)
    return settings[table_name][key_name]

And my variable is defined like this:

is_engine_black: bool = utilities.load_settings("engine", "black")  # Returns: True (so, a bool type)

But the error mypy outputs is this:

mypy: error
assignment - Incompatible types in assignment (expression has type "int", variable has type "bool")

The version of mypy used is 0.991. The operating system used is Windows 11 64-bit (updated to the latest version possible), running on Python 3.11. This error appears in Sublime Text 4 (its latest version of 4143) with the mentioned version of mypy and also in VSCode (also updated to the latest version possible and also updated its mypy extension to the latest version of 0.991 [at the time of writing this]).

So, tell me guys: WTF is going on here?

[hauntsaninja]

You've annotated your function as returning bool | int, so mypy is right to complain. The expression could be int, but your variable is bool.

If your question is why mypy says just int instead of bool | int, that's because bool is a subclass of int at runtime, so bool | int is basically equivalent to int

[PedanticHacker]

So, how should I annotate the return type(s) of my load_settings() function then?

My function can, indeed, return an int if a setting holds an integer value, or it can return a bool if a setting has a value of either True or False. It all depends on what is passed as the table_name and key_name arguments to it.

I know that the bool type is just a subtype of the int type. But annotating only int as a return type of my load_settings() function would be confusing.

What do you suggest?

[hauntsaninja]

Personally, I'd return Any. If you want to be type safe, you could also return a Union like currently, but add asserts. If your names are constants, overloads involving literals can achieve type safety without asserts.
#1693

[PedanticHacker]

overloads involving literals can achieve type safety without asserts

Can you please provide an example what you mean by this?

[gandhis1]

@overload
def load_settings(table_name: Literal["engine"], key_name: Literal["black"]) -> bool: ...

Obviously this doesn't scale well if you have many of these, which is why something like this I'd probably consider returning an object somewhere. Polymorphic return types are a trap in the annotations world.

[PedanticHacker]

Are you saying I should better just annotate the return type of my function as -> int?

def load_settings(table_name: str, key_name: str) -> int: ...

[erictraut]

Here are some alternatives that can help make your code more robust.

You could create separate wrapper functions for bool and int settings, like this:

def load_bool_setting(table_name: str, key_name: str) -> bool:
    setting = load_settings(table_name, key_name)
    if type(setting) != bool:
        raise ValueError(f"key {key_name} was not a bool as expected")
    return setting

Or you could make your original function generic so it handles any setting type:

T = TypeVar("T")

def load_typed_setting(table_name: str, key_name: str, expected_type: type[T]) -> T:
    with open(constants.SETTINGS_FILE_PATH, "rb") as settings_file:
        setting = tomllib.load(settings_file)
    if type(setting) != expected_type:
        raise ValueError(f"key {key_name} was not a {expected_type} as expected")
    return setting

[PedanticHacker]

I have now defined my load_setting() function like this:

def load_setting(group: str, name: str) -> bool | int:
    """Return a setting value by the given group and name."""
    with open("settings.toml", "rb") as settings_file:
        settings = tomllib.load(settings_file)
    return settings[group][name]

The usage of my load_setting() function could be something like this:

load_setting("engine", "pondering")

My settings.toml file looks like this:

[engine]
black = true
white = false
pondering = false

[clock]
time = 60
increment = 0

When I call my load_setting() function, it returns the value False, so the Python bool type representing the value False (and not false as is per TOML syntax stored in my settings.toml file). So, the return type of my load_setting() function is exactly as it should be.

Now, I can't, for the life of me, explain why (oh why?!) does mypy complain about the return type of my load_setting() function as being an int.

However, mypy does not complain if I pass the load_setting() function call to Python's built-in bool() function, so doing this is perfectly fine for mypy:

bool(load_setting("engine", "pondering"))

And only now is mypy finally satisfied. But having to use Python's bool() built-in function is strange. My load_setting() function returns False, but I still need to pass that to bool() just to make mypy to shut up.

Can anyone explain to me why is mypy not capable of handling the union type of a bool and an int?

[erictraut]

Mypy is enforcing the normal rules in the type system. Your function is annotated to return either a bool or an int, but you are attempting to assign it to a value that is declared to be a bool. Mypy is correctly pointing out a potential bug in your code. If your function happens to return an int value and you assign it to a variable that is declared as bool, you are violating the type of that variable. If your intent is for that variable to accept either a bool or an int, you can declare its type to be bool | int (or simply int, since bool is a subtype of int).

If you pass an int value to the constructor of bool, you will convert that value to a valid bool object. But is this really what you want to do? Let's say that the user of your program incorrectly configures his setting file to include the line pondering = 42. If you convert this value to a bool by calling bool(42), you will construct a value of True. Is this what you expect in this situation? More robust code would detect the unexpected value and inform the user that he probably didn't mean to use an integer value for this setting.

[PedanticHacker]

It won't be accurate to annotate the return type of my load_setting() function only as -> bool or only as -> int, because load_setting() can return either a bool or an int.

Let's see two uses of my load_setting() function when it returns a bool and when it returns an int.

load_setting("engine", "pondering")  # The return value is: False (i.e., a bool)
load_setting("clock", "time")  # The return value is: 60 (i.e., an int)

I am passing the result of my load_setting() function to another function which is not from my API but rather from a 3rd-party package, so I have no control over the 3rd-party function that receives the value as returned by my load_setting() function.

But, strangely enough, if I pass my load_setting() function in the form of bool(load_setting("engine", "pondering")) where a bool is expected by a 3rd-party function, mypy does not complain. The same goes for a 3rd-party function that expects an int where passing my load_setting() function in the form of int(load_setting("clock", "time")) does not receive any complaint by mypy.

Why does mypy not complain in the case of bool(load_setting("engine", "pondering")) and/or int(load_setting("clock", "time")) but does complain in the case of load_setting("engine", "pondering") and/or load_setting("clock", "time")?

[ikonst]

It won't be accurate to annotate the return type of my load_setting() function only as -> bool or only as -> int, because load_setting() can return either a bool or an int.

Yes.

I am passing the result of my load_setting() function to another function which is not from my API but rather from a 3rd-party package, so I have no control over the 3rd-party function that receives the value as returned by my load_setting() function.

Yes, this is how it usually is.

Why does mypy not complain in the case of bool(load_setting("engine", "pondering")) and/or int(load_setting("clock", "time")) but does complain in the case of load_setting("engine", "pondering") and/or load_setting("clock", "time")?

bool(anything) turns it into a bool type. For example bool("hello world") is a bool, and bool(my_database_object) is a bool.

Same for int(anything). int("hello world") is an int (even though it'll crash in runtime with TypeError, but it's you and me knowing that, not mypy).

If you had an load_settings which returned int | str, would it be clearer to you why:

value = load_settings("foo", "bar")
third_party_which_expects_str(value)

would detect a typing issue on the 2nd line?

[PedanticHacker]

bool(anything) turns it into a bool type. For example, bool("hello world") is a bool, and bool(my_database_object) is a bool.

Yes, @ikonst, I know this.

I want to know why do I need to use Python's built-in bool() function explicitly to shut up mypy. So, if a 3rd-party function receives a bool type value in the form of False, mypy complains. But if a 3rd-party function receives bool type value in the form of bool(False), mypy does not complain. Why is False not good enough? Why is only bool(False) good enough? Last time I checked: False == bool(False) # True, so they are equivalent. What is going on here?!

[JelleZijlstra]

It sounds like you are essentially doing this:

def some_function(arg: bool): ...

setting: bool | int = ...
some_function(setting)

Mypy obviously should complain about this: an int is not a bool.

[ikonst]

So, if a 3rd-party function receives a bool type value in the form of False, mypy complains.

Your code looks like this (1):

value = load_setting("engine", "pondering")
third_party_which_expects_bool(value)

not like this (2):

third_party_which_expects_bool(False)

Try changing it to (2) and then mypy wouldn't "complain", right?

...

And yes, of course, of course, that's not a change you want to make, since you don't want to pass a constant. You want to pass a value that's only known at runtime. What mypy knows at "check-time" is that it's some value that's either int or bool, and only in the latter case it's OK to pass.

One thing that's confusing here is that an bool is a kind of an int, just like a Cat is a kind of an Animal. If your function returns "a cat or an animal" then you can simply say that it returns "an animal". But then if someone asks you for "a cat", you can't just give them "an animal" since it might be a dog. 🐶

[PedanticHacker]

You want to pass a value that's only known at runtime.

@ikonst, yes, that's the problem with mypy: it can't process types at runtime. But, interestingly enough, whenever you use bool(), then it can process that type at runtime.

I always thought that using type annotations would make my codebase so much more readable, but now I realize I have to fight really hard to adhere to mypy, and that is making my work 10 times harder because I need to painstakingly make sure to make mypy happy in every step. That's like having a very demanding wife. 🤣

[PedanticHacker]

The mypy error I get is this:
Argument "ponder" to "play" of "SimpleEngine" has incompatible type "int"; expected "bool" [arg-type]

The SimpleEngine class is part of the python-chess library, which is a Python library for chess. This class has a play() method with one of its arguments named ponder. This ponder arguments expects a bool type, so either True or False.

As I am passing the result of my load_setting("engine", "pondering") function to the ponder argument, I am hence passing the value False, because my function returns the Python boolean value False (and yes, I have triple-checked that this is indeed the case). I don't understand why mypy thinks an int is being passed. I know the bool type is a subclass of the int type, but mypy shouldn't mistaken a bool for an int.

What is going on here? Does mypy have a bug in not being able to distinguish between a bool and an int?

[ikonst]

mypy doesn't know what your function returns at runtime. mypy knows it returns int or bool, but it doesn't know if it's True, False or 42.

[erictraut]

@PedanticHacker, I think you're confused about the purpose of a static type checker. You may find it useful to read the introduction portion of the mypy documentation.

Static type checkers do not "run" your code. They don't evaluate "values". That's the job of the Python interpreter.

Static type checkers evaluate "types" of expressions within your code and detect potential bugs in your code by looking for type violations. A "type" represent a class of potential values. A static type checker doesn't know (or care) that a particular invocation of a function returns the value False at runtime. It knows that the function returns some value that conforms to a particular type — for example, int | bool. That means the function could return False or True or 0 or 100. If you attempt to assign the return result of such a function to a variable that is declared as accepting only a bool, then it's the job of a static type checker to tell you that you have a type violation (and potential bug) because the value might be an int, and that's not type compatible with bool.

Static type checking is completely optional in Python. If you are not finding value with static type checking, you can simply omit the type annotations and refrain from running mypy.

[PedanticHacker]

Guys, I completely understand what you want to tell me.

But please consider these 2 very important factors:

1. mypy does not produce an error if I pass bool(load_setting("engine", "pondering")) as the ponder argument, so if I am using the built-in bool() function;
2. mypy does produce an error if I pass load_setting("engine", "pondering") as the ponder argument, so if I am not using the built-in bool() function.

Why is using the built-in bool() function okay (i.e., no mypy error occurs) if mypy can't check types at runtime? How is it possible that using bool() makes everything okay? Please explain this to me.

[JelleZijlstra]

Because bool(some int) gives you a bool.

[PedanticHacker]

@JelleZijlstra, please know that the return type of load_setting("engine", "pondering") is False, so a bool type. Then, in my case of using the built-in bool() function, Python has to process this as bool(False), which is the same as it would have to process False [as we all know, bool(False) is False], but for mypy (apparently!) bool(False) is okay in my context, but False is not. Why?

[JelleZijlstra]

The return type is int | bool, not bool. As has been previously explained, static type checkers like mypy look only at the declared return type, not the runtime return value.

[PedanticHacker]

@JelleZijlstra, I explicitly declare that my load_setting() function returns either an int or a bool. Why is mypy so demanding when it comes to a union of these 2 particular types (i.e., int and bool)?

[hauntsaninja]

It's not even necessarily about these two particular types. It's just how Unions in return types work: type checkers correctly assume the worst case that you can return any of the types in the Union.

The following program is buggy. mypy's behaviour ensures that you're alerted to this fact:

class A:
    def only_works_on_a(self) -> None: ...

class B:
    def only_works_on_b(self) -> None: ...

import random

def load_settings() -> A | B:
    # type checker doesn't evaluate the runtime logic, it only looks at the declared return type
    return A() if random.random() < 0.5 else B()

a: A = load_settings()
a.only_works_on_a()  # will fail 50% of the time

[PedanticHacker]

@hauntsaninja, how should I annotate my function, then?

[hauntsaninja]

Myself and others already answered that at the top of this thread ;-) #14255 (comment)

Returning Any would look like:

from typing import Any
def load_settings(key: str) -> Any: ...

Using asserts could look like: #14255 (comment)

Using overload + literals could look like: #14255 (comment)

[PedanticHacker]

Will mypy ever check types at runtime?

[hauntsaninja]

mypy will never run your code.

The Python interpreter will never check types at runtime (unless you write code that does those checks).

[PedanticHacker]

I have a feeling that having -> Any as the return type annotation of my load_setting() function is too broad. I want to express to the reader of my code that my load_setting() function can return either a bool (i.e., True or False) or it can return an int. It all depends what arguments are passed to my load_setting() function.

I immediately come into conflict with mypy when my load_setting() function is passed as an argument to another function that expects only one of those types.

Is there just one and only one obvious way (not three!) to go about it? Something Pythonic, but avoiding the Any type.

[PedanticHacker]

This is still a problem in mypy 1.8.0. How come that -> int | str works fine but -> int | bool doesn't? I know that bool is a subtype of the int type, where int takes precedence, but that's so silly.

I just can't annotate a function/method that returns either an int or a bool without mypy complaining when such a function/method is used.

I have a config.ini file with exactly this contents:

[clock]
time = 60
increment = 0

[engine]
black = true
white = false
pondering = false

And I have a function to get the value of a certain option:

def get_config_value(section: str, option: str) -> int | bool | None:
    """Get the config value of an `option` from the given `section`."""
    config_parser = ConfigParser()
    config_parser.read("rexchess/config.ini")

    if section == "clock":
        return config_parser.getint(section, option)
    elif section == "engine":
        return config_parser.getboolean(section, option)
    else:
        return None

Then I have a property in my Game class...

class Game:
    """An implementation of the standard chess game."""

    def __init__(self) -> None:
        self._engine_plays_as_black: bool = get_config_value("engine", "black")

    @property
    def engine_plays_as_black(self) -> bool:
        return self._engine_plays_as_black

The complaint from mypy I get for the engine_plays_as_black property is this:

mypy: error
return-value - Incompatible return value type (got "int | None", expected "bool")

Can anyone explain to me why is mypy so narrow minded?

[TeamSpen210]

MyPy is correct there. Your function returns int | None, which could be values other than a boolean. I had my own issues with similar code, which I solved by restructuring how the code works. Instead of passing strings into get_config_value(), I created token objects for each setting:

T = TypeVar("T", int, bool, str, float) # etc
class Option(Generic[T]):
	def __init__(self, kind: type[T], section: str, name: str) -> None: ...
	
def get(option: Option[T]) -> T: ....

PLAY_AS_BLACK = Option(bool, "engine", "black")

get(PLAY_AS_BLACK) # bool

In get(), you'd get the option, switching on kind to convert the value appropriately. That bit I don't think can be typesafe, since it needs intersection types for the checker to prove it's correct. But everywhere else is safe, and you don't have to worry about misspelling option names elsewhere in the code. Alternatively, you could stick to how you have it, but pass in the expected type. That's what I did previously:

get(bool, "engine", "black")

[PedanticHacker]

Some time has now passed since I originally asked this question. We now have Python 3.12, which introduces type as a soft keyword. Would there be any benefit by using type to declare an alias, a return type annotation that basically unifies the int and bool types?

So, like this:

type SettingValue = int | bool

I know that type as a soft keyword is not yet recognized by mypy (at the time of me writing this), so I can't test whether this return type annotation is the right approach, which would be -> SettingValue.

I'd like your expert opinion on what you think would happen. Would I stumble upon the same issue as with the return type annotation -> int | bool?

[TeamSpen210]

That won't change anything no. type just produces an alias, it's equivalent to putting the original type hint in directly.

[PedanticHacker]

I understand. So, how would you suggest I go about this? I haven't find any information anywhere regarding an alternative to the return type annotation -> int | bool that would make mypy happy for my particular case.

The main issue is that bool is just a subtype of the int type and not a distinct type in Python. It would be nice if Python made bool a distinct type and not a "hack" type, then my return type annotation would make mypy happy.

It's strange that in Python True + True, for example, is not a TypeError, but a valid int value of 2. There are cases where True being 1 and False being 0 might be a useful shortcut, but I'd prefer True and False being as distinct types as None is.

We need a unique type annotation, something like IntLogical, NumericLogical or IntBool or some appropriate type name, that unifies both int and bool types, which would enable mypy to properly check that if the return type annotation of a function/method is -> int | bool and some other function/method expects a bool, it doesn't nag by saying Incompatible types in assignment (expression has type "int", variable has type "bool").

[TeamSpen210]

The subtype relationship between int/bool isn't relevant actually. The problem is that your function is inherently type-unsafe. A function returning int | bool means that the value could be either of those types. Passing that to a function expecting a bool is invalid, because if the value was an int then that would just be incorrect. Mypy simplifying int | bool ->int happens because those two hints mean exactly the same thing.

Your code right now only works because it knows that certain settings produce integers, and others produce booleans. You need to use generics or overloads to make the return type vary depending on the settings names. One way is to change your API to instead have a bunch of Option objects, which I think is better anyway. That prevents typos with settings names elsewhere in code, and lets you loop over all possible options. Alternatively you could add a lot of overloads to the function using Literal, which might work if the types just depend on section names.

[PedanticHacker]

I think that the most straightforward way would be to create Option types (as suggested by @TeamSpen210) by using the type soft keyword introduced in Python 3.12 (because it is the most readable syntax), but mypy doesn't recognize this as valid syntax yet.

When will mypy recognize type as a soft keyword?

[JelleZijlstra]

When will mypy recognize type as a soft keyword?

There will be provisional support in the next release.

[PedanticHacker]

Excellent! And when will the next release be available?