Type issues with Callables

[jeffrabovsky]

• Are you reporting a bug, or opening a feature request?
  bug

• Please insert below the code you are checking with mypy

from typing import Optional, Iterable, TypeVar, Callable

T = TypeVar('T')


def is_even(n: int) -> bool:
    return n % 2 == 0


# equivalent to https://ruby-doc.org/core-2.5.0/Enumerable.html#method-i-detect
def detect(function: Callable[[T], bool],
           iterable: Iterable[T]) -> Optional[T]:
    for element in iterable:
        if function(element):
            return element
    return None


numbers = [1, 2, 3, 4, 5]

# using a lambda
print(detect(lambda n: n % 2 == 0, numbers))

# using a function
print(detect(is_even, numbers))

• What is the actual behavior/output?

$ mypy detect.py 
detect.py:22: error: Unsupported operand types for % ("object" and "int")
detect.py:25: error: Argument 1 to "detect" has incompatible type "Callable[[int], bool]"; expected "Callable[[object], bool]"
Found 2 errors in 1 file (checked 1 source file)

• What is the behavior/output you expect?
  I was expecting the types to be inferred and the above code to typecheck.

• What are the versions of mypy and Python you are using?

$ mypy --version
mypy 0.770
$ python --version
Python 3.7.7

• Do you see the same issue after installing mypy from Git master?
  yes

$ mypy detect.py
detect.py:22: error: Unsupported operand types for % ("object" and "int")
detect.py:25: error: Argument 1 to "detect" has incompatible type "Callable[[int], bool]"; expected "Callable[[object], bool]"
Found 2 errors in 1 file (checked 1 source file)
$ mypy --version
mypy 0.770+dev.22c67daac7d2b28f33d4e9c7bee4f1a06e61a3e5

• What are the mypy flags you are using? (For example --strict-optional)
  None.

• If mypy crashed with a traceback, please paste the full traceback below.
  It didn't crash

[hauntsaninja]

That's unfortunate. Note that it type checks if you remove the print (or if you upper bound T to int). print takes in object, so there's probably some weird typevar solving going on that causes mypy to then also infer object as the type of T, when instead you'd probably want it infer int (as the meet of object and int).

[JukkaL]

An ad hoc way to fix this could be to not use object in type context to infer type variables.

[Akuli]

I expected inferring type variables to work so that mypy finds all the types that the type variable is expected to work with, and then chooses something sane that works for all of them. If that's not how it works, then how does it work instead?

For example, in print(detect(is_even, numbers)), T has to be a valid argument of is_even, a valid element of numbers and a valid argument of print, so something that works with int, int, object. This should lead to choosing int.

[erictraut]

This appears to be fixed in the latest version of mypy, so I think it can be closed.

[hauntsaninja]

Yeah, unfortunately I think the underlying issue in constraint solving still remains. This was fixed as a side effect of python/typeshed#6314

To reproduce with latest mypy, just define:

def print(
    *values: object, sep: str | None = ..., end: str | None = ..., file = ..., flush: bool = ...
) -> None: ...

It looks like if you go back to ce3975d and just duplicate the existing definition of print as a redundant overload it "fixes" this. My guess is there's something buggy resulting from mypy's notion of type context (similar to pyright's "expected type") and the overload means mypy no longer has object as type context.

Thanks for checking though / print is pretty common, so good news for mypy users!