Crash when iterating a Tuple list of @dataclass Protocol #15618

Kevin-Lovato · 2023-07-07T08:33:26Z

Crash Report

MyPy crashes when trying to validate a code bit enumerating a List[Tuple[MyType, MyType]] where MyType is a @dataclass decorated Protocol (see example code below).

Traceback

(Paths redacted)

kevin.lovato@laptop folder % mypy --show-traceback mypy_test.py
mypy_test.py: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 1.5.0+dev.e0b159e0bb6bd414d2999bfcecbb5432541ec3fd
Traceback (most recent call last):
  File ".../.venv/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File ".../lib/python3.10/site-packages/mypy/__main__.py", line 15, in console_entry
    main()
  File ".../lib/python3.10/site-packages/mypy/main.py", line 94, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File ".../lib/python3.10/site-packages/mypy/main.py", line 173, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File ".../lib/python3.10/site-packages/mypy/build.py", line 195, in build
    result = _build(
  File ".../lib/python3.10/site-packages/mypy/build.py", line 268, in _build
    graph = dispatch(sources, manager, stdout)
  File ".../lib/python3.10/site-packages/mypy/build.py", line 2927, in dispatch
    process_graph(graph, manager)
  File ".../lib/python3.10/site-packages/mypy/build.py", line 3325, in process_graph
    process_stale_scc(graph, scc, manager)
  File ".../lib/python3.10/site-packages/mypy/build.py", line 3420, in process_stale_scc
    mypy.semanal_main.semantic_analysis_for_scc(graph, scc, manager.errors)
  File ".../lib/python3.10/site-packages/mypy/semanal_main.py", line 101, in semantic_analysis_for_scc
    check_type_arguments(graph, scc, errors)
  File ".../lib/python3.10/site-packages/mypy/semanal_main.py", line 385, in check_type_arguments
    with state.wrap_context():
  File ".../.pyenv/versions/3.10.6/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File ".../lib/python3.10/site-packages/mypy/build.py", line 2069, in wrap_context
    yield
  File ".../lib/python3.10/site-packages/mypy/semanal_main.py", line 387, in check_type_arguments
    state.tree.accept(analyzer)
  File ".../lib/python3.10/site-packages/mypy/nodes.py", line 377, in accept
    return visitor.visit_mypy_file(self)
  File ".../lib/python3.10/site-packages/mypy/semanal_typeargs.py", line 59, in visit_mypy_file
    super().visit_mypy_file(o)
  File ".../lib/python3.10/site-packages/mypy/traverser.py", line 114, in visit_mypy_file
    d.accept(self)
  File ".../lib/python3.10/site-packages/mypy/nodes.py", line 788, in accept
    return visitor.visit_func_def(self)
  File ".../lib/python3.10/site-packages/mypy/traverser.py", line 133, in visit_func_def
    self.visit_func(o)
  File ".../lib/python3.10/site-packages/mypy/semanal_typeargs.py", line 65, in visit_func
    super().visit_func(defn)
  File ".../lib/python3.10/site-packages/mypy/mixedtraverser.py", line 37, in visit_func
    super().visit_func(o)
  File ".../lib/python3.10/site-packages/mypy/traverser.py", line 128, in visit_func
    self.visit_var(arg.variable)
  File ".../lib/python3.10/site-packages/mypy/mixedtraverser.py", line 34, in visit_var
    self.visit_optional_type(var.type)
  File ".../lib/python3.10/site-packages/mypy/mixedtraverser.py", line 112, in visit_optional_type
    t.accept(self)
  File ".../lib/python3.10/site-packages/mypy/types.py", line 1413, in accept
    return visitor.visit_instance(self)
  File ".../lib/python3.10/site-packages/mypy/semanal_typeargs.py", line 125, in visit_instance
    self.validate_args(info.name, t.args, info.defn.type_vars, t)
  File ".../lib/python3.10/site-packages/mypy/semanal_typeargs.py", line 174, in validate_args
    if not is_subtype(arg, tvar.upper_bound):
  File ".../lib/python3.10/site-packages/mypy/subtypes.py", line 177, in is_subtype
    return _is_subtype(left, right, subtype_context, proper_subtype=False)
  File ".../lib/python3.10/site-packages/mypy/subtypes.py", line 332, in _is_subtype
    return left.accept(SubtypeVisitor(orig_right, subtype_context, proper_subtype))
  File ".../lib/python3.10/site-packages/mypy/types.py", line 2299, in accept
    return visitor.visit_tuple_type(self)
  File ".../lib/python3.10/site-packages/mypy/subtypes.py", line 757, in visit_tuple_type
    mypy.typeops.tuple_fallback(left), right
  File ".../lib/python3.10/site-packages/mypy/typeops.py", line 121, in tuple_fallback
    return Instance(info, [join_type_list(items)], extra_attrs=typ.partial_fallback.extra_attrs)
  File ".../lib/python3.10/site-packages/mypy/join.py", line 674, in join_type_list
    joined = join_types(joined, t)
  File ".../lib/python3.10/site-packages/mypy/join.py", line 252, in join_types
    return t.accept(TypeJoinVisitor(s, instance_joiner))
  File ".../lib/python3.10/site-packages/mypy/types.py", line 1413, in accept
    return visitor.visit_instance(self)
  File ".../lib/python3.10/site-packages/mypy/join.py", line 329, in visit_instance
    if t.type.is_protocol and is_protocol_implementation(self.s, t):
  File ".../lib/python3.10/site-packages/mypy/subtypes.py", line 1049, in is_protocol_implementation
    supertype = get_proper_type(find_member(member, right, left))
  File ".../lib/python3.10/site-packages/mypy/subtypes.py", line 1137, in find_member
    return find_node_type(method, itype, subtype, class_obj=class_obj)
  File ".../lib/python3.10/site-packages/mypy/subtypes.py", line 1271, in find_node_type
    itype = map_instance_to_supertype(itype, node.info)
  File ".../lib/python3.10/site-packages/mypy/maptype.py", line 19, in map_instance_to_supertype
    if superclass.fullname == "builtins.tuple" and instance.type.tuple_type:
  File ".../lib/python3.10/site-packages/mypy/nodes.py", line 3409, in __getattribute__
    raise AssertionError(object.__getattribute__(self, "msg"))
AssertionError: FuncBase for non-methods lack info
mypy_test.py: : note: use --pdb to drop into pdb

To Reproduce

I used the trunk version of MyPy and ran mypy --show-traceback mypy_test.py using the following code snippet:

from dataclasses import dataclass
from datetime import date
from typing import Protocol, List, Tuple


class ValidityPeriod:
    start_date: date
    end_date: date | None


@dataclass
class WithValidityPeriod(Protocol):
    validity_period: ValidityPeriod

    def do_overlap(self, other: "WithValidityPeriod") -> bool: ...


def do_periods_overlap(periods: List[Tuple["WithValidityPeriod", "WithValidityPeriod"]]) -> bool:
    for period_1, period_2 in periods:
        if period_1.do_overlap(period_2):
            return True
    return False

Your Environment

Mypy version used: 1.5.0+dev.e0b159e0bb6bd414d2999bfcecbb5432541ec3fd
Mypy command-line flags: none
Mypy configuration options from mypy.ini (and other config files): none
Python version used: 3.10.6
Operating system and version: MacOS Ventura 13.4

The text was updated successfully, but these errors were encountered:

AlexWaygood · 2023-07-07T08:38:31Z

This is a valid crash report (since mypy should never crash on syntactically valid Python code). But the quick fix is "don't do that" — it doesn't really make much sense to add @dataclass to a protocol :)

Kevin-Lovato · 2023-07-07T08:50:17Z

This is a valid crash report (since mypy should never crash on syntactically valid Python code). But the quick fix is "don't do that" — it doesn't really make much sense to add @dataclass to a protocol :)

For context I believe the original writer of this code wanted to offer an API using a Protocol but then also added features in it (the example is shortened) and ended up having some classes inheriting from the Protocol.

I agree that the use case is odd (and I plan on refactoring it), but the PEP shows examples of explicitly inheriting a Protocol so it could be a "valid" usage.

Thanks for the swift answer :)

AlexWaygood · 2023-07-07T08:52:36Z

Interestingly, I can only reproduce on mypy master, not with mypy 1.4.1. Can you reproduce with mypy 1.4.1 as well?

AlexWaygood · 2023-07-07T08:57:16Z

Anyway, here's a smaller repro that crashes with the same traceback on the master branch:

from dataclasses import dataclass
from typing import Protocol, Tuple


@dataclass
class WithValidityPeriod(Protocol): ...

periods: Tuple[WithValidityPeriod, WithValidityPeriod]
list(periods)

AlexWaygood · 2023-07-07T10:02:40Z

I bisected to find the source of the crash:

6f2bfff521e708f015af1eec5118db4600b829be is the first bad commit
commit 6f2bfff521e708f015af1eec5118db4600b829be
Author: Ilya Priven <ilya.konstantinov@gmail.com>
Date:   Sat Jun 17 09:02:36 2023 -0400

    Add signature for dataclasses.replace (#14849)

(Commit 6f2bfff - PR #14849)

cc. @ikonst

Kevin-Lovato · 2023-07-07T14:59:30Z

Interestingly, I can only reproduce on mypy master, not with mypy 1.4.1. Can you reproduce with mypy 1.4.1 as well?

We had multiple issues so it's possible that what crashed on 1.4.1 and led me to using master was a different issue, my bad.

Thanks for the simple repro bit.

AlexWaygood · 2023-07-07T15:00:59Z

Interestingly, I can only reproduce on mypy master, not with mypy 1.4.1. Can you reproduce with mypy 1.4.1 as well?

We had multiple issues so it's possible that what crashed on 1.4.1 and led me to using master was a different issue, my bad.

It's still a very valid crash report, and it's great to discover these things before a release!

ikonst · 2023-07-07T15:05:17Z

I'll take a look over the weekend.

The root cause is hacky creation of incomplete symbols; instead switching to `add_method_to_class` which does the necessary housekeeping. Fixes #15618.

Kevin-Lovato added the crash label Jul 7, 2023

AlexWaygood added topic-protocols topic-dataclasses labels Jul 7, 2023

ikonst mentioned this issue Jul 9, 2023

Fix dataclass/protocol crash on joining types #15629

Merged

JukkaL closed this as completed in #15629 Jul 14, 2023

JukkaL pushed a commit that referenced this issue Jul 14, 2023

Fix dataclass/protocol crash on joining types (#15629)

7a94183

The root cause is hacky creation of incomplete symbols; instead switching to `add_method_to_class` which does the necessary housekeeping. Fixes #15618.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Crash when iterating a Tuple list of @dataclass Protocol #15618

Crash when iterating a Tuple list of @dataclass Protocol #15618

Kevin-Lovato commented Jul 7, 2023

AlexWaygood commented Jul 7, 2023

Kevin-Lovato commented Jul 7, 2023

AlexWaygood commented Jul 7, 2023

AlexWaygood commented Jul 7, 2023 •

edited

AlexWaygood commented Jul 7, 2023 •

edited

Kevin-Lovato commented Jul 7, 2023

AlexWaygood commented Jul 7, 2023

ikonst commented Jul 7, 2023

Crash when iterating a Tuple list of @dataclass Protocol #15618

Crash when iterating a Tuple list of @dataclass Protocol #15618

Comments

Kevin-Lovato commented Jul 7, 2023

AlexWaygood commented Jul 7, 2023

Kevin-Lovato commented Jul 7, 2023

AlexWaygood commented Jul 7, 2023

AlexWaygood commented Jul 7, 2023 • edited

AlexWaygood commented Jul 7, 2023 • edited

Kevin-Lovato commented Jul 7, 2023

AlexWaygood commented Jul 7, 2023

ikonst commented Jul 7, 2023

AlexWaygood commented Jul 7, 2023 •

edited

AlexWaygood commented Jul 7, 2023 •

edited