Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Internal error on processing NamedTuple with method and recursive field #8695

Closed
michalbednarski opened this issue Apr 18, 2020 · 7 comments
Closed
Labels

Comments

@michalbednarski
Copy link

Checking following code with mypy results in INTERNAL ERROR: maximum semantic analysis iteration count reached along with Deferral trace cycling with lines 7, 7 and -1

from typing import NamedTuple

class First(NamedTuple):
    def method(self):
        pass

class Second(NamedTuple):
    recursive: 'Second'

As far as I've debugged mypy code I think it is caused by analyze_class_body_common being called under save_namedtuple_body contextmanager which sets empty SymbolTable for class.
Then analyze_class_body_common processes methods defined in class body, method definition is processed through SemanticAnalyzer.visit_func_def which calls add_function_to_symbol_table.
Ultimately add_symbol_table_node upon not seeing that method was already handled (because we've cleared SymbolTable), sets SemanticAnalyzer#progress = True on every analysis iteration, preventing process_top_levels from setting final_iteration which breaks analysis cycle

@ndevenish
Copy link

I just ran into this, and reduced it to an amusingly identical test case, except for the fact that this also happens with a # type: ignore marker to avoid an error about the recursive type:

from __future__ import annotations

from typing import NamedTuple

class ScopeName(NamedTuple):
    def somefun(self):
        pass

class Scope(NamedTuple):
    children: Scope  # type: ignore

Presumably with this marker mypy is still checking the type and just silencing the warnings. It looks like the only way to avoid this is to remove the annotation completely?

Checked on master (781dd69) python 3.7/3.8.

@eyqs
Copy link

eyqs commented Aug 7, 2020

This may be causing downstream bugs in the django-stubs project, as well, since the django-stubs Mypy plugin crashes when I use NamedTuple in this minimal reproducible example.

@habilain
Copy link

habilain commented Sep 2, 2020

This might be classed as "not a bug" or a low priority bug by the MyPy due to the impossibility of instantiating the example (i.e. if the recursive attribute of Second is always an instance of Second, you can never actually instantiate Second correctly - you need to have another case to be able to terminate the chain of Second's). However, there does appear to be a bug in the cycle detector because if we change the recursive attribute to be Optional (to make it possible to instantiate), we have the following odd behaviour:

from __future__ import annotations
from typing import NamedTuple, Optional

class First(NamedTuple):
    def method(self): ...

class Second(NamedTuple):
    recursive: Optional[Second]

Causes MyPy to crash with "mypy-sample-4.py: error: INTERNAL ERROR: maximum semantic analysis iteration count reached"

from __future__ import annotations
from typing import NamedTuple, Optional

class Second(NamedTuple):
    recursive: Optional[Second]

Causes MyPy to fail with "mypy-sample-4.py:8: error: Cannot resolve name "Second" (possible cyclic definition)". So that means if you have a NamedTuple with a method, it changes the behaviour of the cycle detector on a different, unrelated NamedTuple - that certainly seems wrong!

Note that I'd argue both of these behaviours are wrong as a frozen dataclass will handle this just fine (see other issue mentioned above for that bug)

@altvod
Copy link

altvod commented Dec 4, 2020

I have the exact same issue.
If simplified, my code comes down to this:

from typing import Tuple, NamedTuple


class Tree(NamedTuple):
    children: Tuple['Tree', ...] = ()  # type: ignore


class ClassThatDoesSomething(NamedTuple):
    def do_something(self) -> None:
        return None

Took me a while to isolate this...

It used to work in earlier versions (don't know exactly when it broke).

It's a bit frustrating because we (my team and I) have a huge project that used to be ok, but now fails to be checked by mypy.

@ramalho
Copy link

ramalho commented Jul 16, 2021

This may be a manifestation of another bug: #9397, unrelated to #731

@JelleZijlstra
Copy link
Member

The OP's example still fails on 0.942. Output:

Deferral trace:
    __main__:7
    __main__:7
    __main__:-1
    __main__:7
    __main__:7
    __main__:-1
    __main__:7
    __main__:7
    __main__:-1
    __main__:7
    __main__:7
    __main__:-1
    __main__:7
    __main__:7
    __main__:-1
    __main__:7
    __main__:7
    __main__:-1
    __main__:7
    __main__:7
    __main__:-1
    __main__:7
    __main__:7
    __main__:-1
    __main__:7
    __main__:7
    __main__:-1
    __main__:7
    __main__:7
    __main__:-1
    __main__:7
    __main__:7
    __main__:-1
    __main__:7
    __main__:7
    __main__:-1
    __main__:7
    __main__:7
    __main__:-1
    __main__:7
    __main__:7
    __main__:-1
    __main__:7
    __main__:7
    __main__:-1
    __main__:7
    __main__:7
    __main__:-1
    __main__:7
    __main__:7
    __main__:-1
    __main__:7
    __main__:7
    __main__:-1
    __main__:7
    __main__:7
    __main__:-1
    __main__:7
    __main__:7
    __main__:-1
main.py: error: INTERNAL ERROR: maximum semantic analysis iteration count reached
Found 1 error in 1 file (errors prevented further checking)

Interestingly it doesn't crash when the First class is removed; then we just get the error from #9397.

@ilevkivskyi
Copy link
Member

The original example works on master with --enable-recursive-aliases.

melton-jason added a commit to specify/specify7 that referenced this issue Apr 5, 2023
More specifically, our version of mypy has a known issue regarding
processing the type hints for subclasses of NamedTuples which
reference themselves.
i.e., using the "DeferredScopeUploadTable" type hint in the
DeferredScopeUploadTable class

For more details, see
python/mypy#8695
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

9 participants