ambiguity on using union of type[generic type] and callable in mypy==0.981

[Yun-Kim]

Bug Report
I have a function which expects either a callable function or a type of generic type (i.e. float, str, int, ...) as an argument. However in mypy == 0.981, the mypy type checker strictly assumes that a type of type is a callable first and foremost (to be fair, it is technically a callable since it can also be used as a constructor). This results in an error even if I've passed in a type of type as I've intended.

To Reproduce

Here is my code that is erroring:

from typing import Callable, Type, TypeVar, Union


T = TypeVar("T")


def accepts_type_or_callable(name, value, value_type_or_callable):
    # type: (str, T, Union[Type[T], Callable[[Union[str, T, None]], T]]) -> T
    return value


accepts_type_or_callable("Test Name", 14.5, float)

Expected Behavior
No errors. I don't see any errors when using mypy <= 0.971.

Actual Behavior
When using mypy == 0.981, I get this mypy error:

error: Argument 3 to "accepts_type_or_callable" has incompatible type "Type[float]"; expected "Callable[[Union[str, float, None]], float]"

For some reason, mypy expects a float instead of Type[float] as I've specified. I'd love to get some insight on why it changes this expectation.

Your Environment

• Mypy version used: 0.981
• Mypy command-line flags: None
• Mypy configuration options from mypy.ini (and other config files): None
• Python version used: N/A

[hauntsaninja]

Thanks for the report! mypy_primer -p ~/dev/mypy_primer/test.py --bisect --new v0.981 --old v0.971 --debug bisects this to #13303. Unlike #13750, this one still reproes on master.

[ilevkivskyi]

Yeah, unfortunately Type[T] vs Callable[..., T] is a known problematic area. The rules in PEP 484 introduced some ambiguities in semantics, and mypy was actually never really principled about it. I tried to clean it up a bit, but some edge cases were not really clear.

IIRC this "rule" (Type[T] is a proper subtype of Callable[P, T] for all P) is tied closely to overload checking, I will how to make semantics here more reasonable.

[Yun-Kim]

Thanks for getting back to me so quick and letting me know!

Do you have any timeline on when you might look into making the semantics here more reasonable? Just for reference so I know roughly when to remove my type ignores 😄

[ilevkivskyi]

It depends, but maybe as soon as this weekend.

[jace]

I've found a variant of this affecting overload declarations. Sample code:

https://mypy-play.net/?mypy=latest&python=3.11&gist=8d1b1a7dc948f7ad4da6b26b646ab5b6

The core issue is that these overloads overlap:

@overload
def decorator(decorated: Type[T]) -> Type[T]: ...

@overload
def decorator(decorated: Callable[Concatenate[Any, P], R]) -> WrapperClass[P, R]: ...

[jace]

I was able to resolve this by using a callback protocol, as #14121 explicitly excludes type.__call__ when matching Protocol.__call__. For anyone else who stumbles on this:

T = TypeVar('T')
P = ParamSpec('P')
R = TypeVar('R')

class MethodProtocol(Protocol[P, R]):
    # Using ``def __call__`` seems to break Mypy (1.7.1), so we use this hack:
    # https://github.com/python/typing/discussions/1312#discussioncomment-4416217
    __call__: Callable[Concatenate[Any, P], R]

@overload
def decorator(decorated: Type[T]) -> Type[T]: ...

@overload
def decorator(decorated: MethodProtocol[Concatenate[Any, P], R]) -> WrapperClass[P, R]: ...

...

[jace]

Update: As of Mypy 1.9.0, the hacky use of Callable within Protocol is no longer required. This works now:

T = TypeVar('T')
P = ParamSpec('P')
R_co = TypeVar('R_co', covariant=True)

class MethodProtocol(Protocol[P, R_co]):
    def __call__(__self, self: Any, *args: P.args, **kwargs: P.kwargs) -> R_co: ...

@overload
def decorator(decorated: Type[T]) -> Type[T]: ...

@overload
def decorator(decorated: MethodProtocol[P, R_co]) -> WrapperClass[P, R_co]: ...