mypy failed to evaluate TypeVar in function parameter

[KaitoHH]

Bug Report

mypy cannot infer typevar which is defined as the function parameter, even though it could be evaluated from the following decorated function

To Reproduce

import functools
import time
from typing import Callable, ParamSpec, TypeVar, Any

_P = ParamSpec("_P")
_T = TypeVar("_T")


def retry(
    max_retry: int = 3,
    interval: int = 1,
    should_retry: Callable[[_T], bool] | None = None,
) -> Callable[[Callable[_P, _T]], Callable[_P, _T]]:
    def decorator(func: Callable[_P, _T]) -> Callable[_P, _T]:
        @functools.wraps(func)
        def wrapper(*args: _P.args, **kwargs: _P.kwargs) -> _T:
            failed_cnt = 0
            while True:
                try:
                    result = func(*args, **kwargs)
                    if should_retry is not None and should_retry(result):
                        raise ValueError("result is not expected")
                    return result
                except Exception as e:
                    failed_cnt += 1
                    if failed_cnt > max_retry:
                        e.add_note(f"Retry {max_retry} times failed.")
                        raise e
                    time.sleep(interval)

        return wrapper

    return decorator

@retry() # Argument 1 has incompatible type "Callable[[int], int]"; expected "Callable[[int], Never]"
def f(x: int) -> int:
    return x


if __name__ == "__main__":
    f(1)

Expected Behavior

no error

Actual Behavior

Argument 1 has incompatible type "Callable[[int], int]"; expected "Callable[[int], Never]"

Your Environment

• Mypy version used: 1.15.0
• Python version used: 3.12

[sterliakov]

The high-level problem is that _T is bound at the first call site (retry()) if should_retry passed, but only at the application site (retry()(...)) otherwise. Since _T is clearly present in args of your function, mypy always picks the first way.

I'm not even sure this is a bug. Pyright also solves _T to Unknown in your snippet.

As a workaround, you can define overloaded signatures, so that retry() without should_retry does not contain _T in the arguments types: https://mypy-play.net/?mypy=master&python=3.12&flags=strict&gist=1fecf44b1a447f66c51de6ac7276e013

@overload
def retry(
    max_retry: int = 3,
    interval: int = 1,
    should_retry: None = None,
) -> Callable[[Callable[_P, _T]], Callable[_P, _T]]: ...
@overload
def retry(
    max_retry: int = 3,
    interval: int = 1,
    should_retry: Callable[[_T], bool] = ...
) -> Callable[[Callable[_P, _T]], Callable[_P, _T]]: ...
def retry(
    max_retry: int = 3,
    interval: int = 1,
    should_retry: Callable[[_T], bool] | None = None,
) -> Callable[[Callable[_P, _T]], Callable[_P, _T]]:
    ...

[KaitoHH]

The high-level problem is that _T is bound at the first call site (retry()) if should_retry passed, but only at the application site (retry()(...)) otherwise. Since _T is clearly present in args of your function, mypy always picks the first way.

I'm not even sure this is a bug. Pyright also solves _T to Unknown in your snippet.

As a workaround, you can define overloaded signatures, so that retry() without should_retry does not contain _T in the arguments types: https://mypy-play.net/?mypy=master&python=3.12&flags=strict&gist=1fecf44b1a447f66c51de6ac7276e013

@overload
def retry(
    max_retry: int = 3,
    interval: int = 1,
    should_retry: None = None,
) -> Callable[[Callable[_P, _T]], Callable[_P, _T]]: ...
@overload
def retry(
    max_retry: int = 3,
    interval: int = 1,
    should_retry: Callable[[_T], bool] = ...
) -> Callable[[Callable[_P, _T]], Callable[_P, _T]]: ...
def retry(
    max_retry: int = 3,
    interval: int = 1,
    should_retry: Callable[[_T], bool] | None = None,
) -> Callable[[Callable[_P, _T]], Callable[_P, _T]]:
    ...

Thank you so much for the solution. Actually I had previously suspected that without should_retry args, if we only defined foo=retry(), the type of foo remains unknown, which causes the complaint of mypy, so I guessed that causes the problem.

Anyway thanks for the reply.