false positive: functools.lru_cache() breaks assert_type(..., SomeEnum)

[finite-state-machine]

Bug Report

Interactions between @functools.lru_cache and Enums confuse assert_type(), resulting in a false positives.

Return values from functions which are decorated by functools.lru_cache() and have return type SomeEnum are treated as Any by assert_type(), while return values from undecorated functions work correctly.

In all cases, reveal_type() reports the expected values, even as assert_type() insists the values are of type Any.

Steps to reproduce

In the following code, method1() exhibits the issue while the nearly-identical method2() does not.

This code is also available on mypy-play.net.

from __future__ import annotations

import functools
import random
from enum import (
        auto,
        Enum,
        )
from typing import (
        Literal,
        )

from typing_extensions import (
        assert_type,
        reveal_type,
        )

class SomeEnum(Enum):
    A = auto()
    B = auto()

def method1() -> None:
    '''this method demonstrates the issue
    '''
    some_enum: SomeEnum = _inner_method1()
    reveal_type(some_enum)  # 'SomeEnum'
    assert_type(some_enum, SomeEnum)
            # error: Expression is of type "Any", not "SomeEnum"
            #        [assert-type]
    reveal_type(some_enum)  # 'SomeEnum'

    # the type can even be manipulated as expected (according to
    # 'reveal_type()') but continues to misbehave with 'assert_type()':

    if some_enum is SomeEnum.A:
        return
    reveal_type(some_enum)  # 'Literal[SomeEnum.B]'
    assert_type(some_enum, Literal[SomeEnum.B])
            # error: Expression is of type "Any", not "Literal[SomeEnum.B]"
            #        [assert-type]
    reveal_type(some_enum)  # 'Literal[SomeEnum.B]'

@functools.lru_cache()
def _inner_method1() -> SomeEnum:
    '''identical to '_inner_method2()', except that it IS decorated
       with '@functools.lru_cache'
    '''
    if random.random() < 0.50:
        return SomeEnum.A
    return SomeEnum.B

def method2() -> None:  # no issues
    '''identical to 'method1()' except that it calls '_inner_method2()'
       instead of '_inner_method1()'
    '''
    some_enum: SomeEnum = _inner_method2()
    reveal_type(some_enum)  # 'SomeEnum'
    assert_type(some_enum, SomeEnum)
            # no error
    reveal_type(some_enum)  # 'SomeEnum'

    if some_enum is SomeEnum.A:
        return
    reveal_type(some_enum)  # 'Literal[SomeEnum.B]'
    assert_type(some_enum, Literal[SomeEnum.B])
            # no error
    reveal_type(some_enum)  # 'Literal[SomeEnum.B]'

def _inner_method2() -> SomeEnum:
    '''identical to '_inner_method1()' except that it's NOT decorated
       with '@functools.lru_cache'
    '''
    if random.random() < 0.50:
        return SomeEnum.A
    return SomeEnum.B

diff

A side-by-side diff of the method1() and method2() sections:

Accessible format of diff

--- one	2023-08-11 08:06:53
+++ two	2023-08-11 08:06:57
@@ -1,30 +1,24 @@
-def method1() -> None:
-    '''this method demonstrates the issue
+def method2() -> None:  # no issues
+    '''identical to 'method1()' except that it calls '_inner_method2()'
+       instead of '_inner_method1()'
     '''
-    some_enum: SomeEnum = _inner_method1()
+    some_enum: SomeEnum = _inner_method2()
     reveal_type(some_enum)  # 'SomeEnum'
     assert_type(some_enum, SomeEnum)
-            # error: Expression is of type "Any", not "SomeEnum"
-            #        [assert-type]
+            # no error
     reveal_type(some_enum)  # 'SomeEnum'
 
-    # the type can even be manipulated as expected (according to
-    # 'reveal_type()') but continues to misbehave with 'assert_type()':
-
     if some_enum is SomeEnum.A:
         return
     reveal_type(some_enum)  # 'Literal[SomeEnum.B]'
     assert_type(some_enum, Literal[SomeEnum.B])
-            # error: Expression is of type "Any", not "Literal[SomeEnum.B]"
-            #        [assert-type]
+            # no error
     reveal_type(some_enum)  # 'Literal[SomeEnum.B]'
 
-@functools.lru_cache()
-def _inner_method1() -> SomeEnum:
-    '''identical to '_inner_method2()', except that it IS decorated
+def _inner_method2() -> SomeEnum:
+    '''identical to '_inner_method1()' except that it's NOT decorated
        with '@functools.lru_cache'
     '''
     if random.random() < 0.50:
         return SomeEnum.A
     return SomeEnum.B

Expected Behavior

Aside from notes arising from use of reveal_type(), there should be no mypy output.

Actual Behavior

Decorating _inner_method1() with functools.lru_cache() causes this issue to manifest. Removing the decoration result in the expected behaviour.

After removing notes arising from reveal_type(), the output is (mypy 1.4.1, Python 3.11):

# method1:
main.py:27: error: Expression is of type "Any", not "SomeEnum"  [assert-type]
main.py:38: error: Expression is of type "Any", not "Literal[SomeEnum.B]"  [assert-type]
# method2:
# (silent, as expected)

Your Environment

• Mypy version used: 1.5.0, trunk (2023-08-11)
  • the issue exists at least as far back as mypy 0.950; earlier versions lack assert_type()
• Mypy command-line flags: (none)
• Mypy configuration options from mypy.ini (and other config files): (none)
• Python version used: 3.8, 3.11

[AlexWaygood]

@JelleZijlstra, are you sure this one's a typeshed issue? It looks like mypy's revealing the correct type with the reveal_type() calls, but is then giving a different answer when you try to assert that type using assert_type(). That sounds like a mypy issue to me

[Akuli]

I minimized it a bit:

from __future__ import annotations

import sys
from enum import auto, Enum
from typing import Callable, TypeVar, Generic
_T = TypeVar("_T")

from typing_extensions import assert_type, reveal_type

class SomeEnum(Enum):
    A = auto()
    B = auto()

class _lru_cache_wrapper(Generic[_T]):
    def __call__(self) -> _T:
        assert False

def lru_cache(f: Callable[..., _T]) -> _lru_cache_wrapper[_T]:
    assert False

@lru_cache
def _inner_method1() -> SomeEnum:
    return SomeEnum.A

def method1() -> None:
    some_enum: SomeEnum = _inner_method1()
    reveal_type(some_enum)            # 'SomeEnum'
    assert_type(some_enum, SomeEnum)  # error
    reveal_type(some_enum)            # 'SomeEnum'

[finite-state-machine]

(@Akuli: Thanks for this! I'll try to include a minimal reproduction case in any future issues.)

[AlexWaygood]

I wasn't able to repro the issue on mypy playground with @Akuli's minimal repro, but I can with this variation on it (which is also slightly more reduced):

from enum import Enum
from typing import Callable, TypeVar
from typing_extensions import assert_type

T = TypeVar("T")

def lru_cache(f: Callable[[], T]) -> Callable[[], T]:
    return f

class SomeEnum(Enum):
    A = object()
    B = object()

def method1() -> None:
    some_enum: SomeEnum = _inner_method1()
    reveal_type(some_enum)              # 'SomeEnum'
    assert_type(some_enum, SomeEnum)    # error: Expression is of type "Any", not "SomeEnum" [assert-type]

@lru_cache
def _inner_method1() -> SomeEnum:
    return SomeEnum.A

https://mypy-play.net/?mypy=latest&python=3.11&gist=81326668cbc8056d8f3b44d3396ea0d0

[finite-state-machine]

Thanks so much, @sobolevn, @Akuli, @AlexWaygood, @JelleZijlstra (and anyone I may have missed)!