too wide source positions for exceptions during pattern matching (3.11.0rc2)

[15r10nk]

The source range in the traceback is to wide in some cases, if an exception is thrown during matching of a pattern.

The Ranges

example for attribute lookup:

class A:
    def __getattr__(self, name):
        assert name[0] == "a"
        return 5


match A():
    case A(apple=5, banana=7):
        print("hi")

output (Python 3.11.0rc2+):

Traceback (most recent call last):
  File "/home/frank/projects/cpython/match_test.py", line 8, in <module>
    case A(apple=5, banana=7):
         ^^^^^^^^^^^^^^^^^^^^
  File "/home/frank/projects/cpython/match_test.py", line 3, in __getattr__
    assert name[0] == "a"
AssertionError

Annotating only the source positions where the attribute check failed (banana=7) would make the traceback easier to read.

The same problem exists for positional matching:

class A:
    __match_args__ = ("apple", "banana")

    def __getattr__(self, name):
        assert name == "apple"


match A():
    case A(5, 7):
        print("hi")

output (Python 3.11.0rc2+):

Traceback (most recent call last):
  File "/home/frank/projects/cpython/match_test2.py", line 9, in <module>
    case A(5, 7):
         ^^^^^^^
  File "/home/frank/projects/cpython/match_test2.py", line 5, in __getattr__
    assert name == "apple"
AssertionError

Equality checks are handled better:

class NeverEqual:
    def __eq__(self, other):
        assert other==5


class A:
    def __init__(self):
        self.a = NeverEqual()
        self.b = 3


match A():
    case A(a=5|3, b=3):
        print("hi")

output (Python 3.11.0rc2+):

Traceback (most recent call last):
  File "/home/frank/projects/cpython/match_test3.py", line 13, in <module>
    case A(a=5|3, b=3):
               ^
  File "/home/frank/projects/cpython/match_test3.py", line 3, in __eq__
    assert other==5
AssertionError

Instance checks are represented in the same way:

class MetaB(type):
    def __instancecheck__(self, instance):
        assert False, "do not use me"


class B(metaclass=MetaB):
    pass


class A:
    def __init__(self):
        self.a = 5
        self.b = 3


match A():
    case A(a=5, b=2|B()):
        print("hi")

output (Python 3.11.0rc2+):

Traceback (most recent call last):
  File "/home/frank/projects/cpython/match_test4.py", line 17, in <module>
    case A(a=5, b=2|B()):
                    ^^^
  File "/home/frank/projects/cpython/match_test4.py", line 3, in __instancecheck__
    assert False, "do not use me"
AssertionError: do not use me

Problem for the User

Consistent source ranges would improve the readability of the tracebacks.

Is it possible to highlight always only the failing Pattern (with attribute name if given) for failing attribute access?

example:

case A(a=5):
       ^^^
case A(a=B()):
       ^^^^^
case A(5):
       ^
case A(5|B()):
       ^^^^^

Instance checks and equality tests are already handled well:

case A(a=5):
         ^
case A(a=B()):
         ^^^
case A(5):
       ^
case A(B()):
       ^^^
case A(5|B())
         ^^^

Problem for libraries

Libraries are using the source ranges for all sorts of functionality.
Executing for example is using this ranges (for the new 3.11 support) to map from instructions back to ast-nodes. Useful ranges are here required to map to useful ast-nodes.

[15r10nk]

I just saw that the arguments in the MatchClass ast-node have no node for the argument name with the value.

case A(a=5):
       ^^^ # there is no ast-node

This would make it really difficult for cpython and libraries to work with such ranges.

I don't know what the best solution might be.

[brandtbucher]

Yeah, this could certainly be tightened up a bit. It definitely falls under the general theme of #93691, so not a super high priority at the moment.

[iritkatriel]

For the first example (in #96999 (comment)), the exception comes from a MATCH_CLASS bytecode that covers the whole range. Unless we break this up into finer bytecodes, I don't know if there is a way to associate smaller ranges with the errors.

[markshannon]

The locations for most, if not all, other statements and expressions have been narrowed and fixed, so it would make sense to fix this as well.

Breaking the MATCH_CLASS bytecode into lower level ops fits in with our general strategy of using a more consistent level of tier 1 instructions that can be specialized and/or handled effectively by the JIT.