Segfault in new PEG parser

[stestagg]

BPO	40903
Nosy	@vstinner, @pablogsal, @miss-islington, @stestagg
PRs	bpo-40903: Handle multiple '=' in invalid assignment rules in the PEG parser #20697 [3.9] bpo-40903: Handle multiple '=' in invalid assignment rules in the PEG parser (GH-20697) #20704

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/pablogsal'
closed_at = <Date 2020-06-08.10:04:06.100>
created_at = <Date 2020-06-07.22:09:09.431>
labels = ['interpreter-core', '3.10', 'type-crash']
title = 'Segfault in new PEG parser'
updated_at = <Date 2020-06-08.16:04:00.152>
user = 'https://github.com/stestagg'

bugs.python.org fields:

activity = <Date 2020-06-08.16:04:00.152>
actor = 'pablogsal'
assignee = 'pablogsal'
closed = True
closed_date = <Date 2020-06-08.10:04:06.100>
closer = 'pablogsal'
components = ['Interpreter Core']
creation = <Date 2020-06-07.22:09:09.431>
creator = 'stestagg'
dependencies = []
files = []
hgrepos = []
issue_num = 40903
keywords = ['patch']
message_count = 6.0
messages = ['370916', '370948', '370968', '371002', '371003', '371004']
nosy_count = 4.0
nosy_names = ['vstinner', 'pablogsal', 'miss-islington', 'stestagg']
pr_nums = ['20697', '20704']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue40903'
versions = ['Python 3.10']

[stestagg]

The input p=p= causes python 3.10 to crash.

I bisected the change, and the behavior appears to have been introduced by 16ab070 (bpo-40334: Correctly identify invalid target in assignment errors (GH-20076) )

Steps to reproduce:

$ echo 'p=p=' | /path/to/python3.10
=== SIGSEGV (Address boundary error)

Analysis:

This code is an invalid assignment, and the parser tries to generate a useful message for this case (invalid_assignment_rule).

However, the target of the assignment is a Name node.

The invalid_assignment_rule function tries to identify the target of the assignment, to create a useful description for the error menssage by calling _PyPegen_get_invalid_target, passing in the Name Node.

PyPegen_get_invalid_target returns NULL if the type is a Name type (pegen.c:2114).

The result of this call is then passed unconditionally to _PyPegen_get_expr_name, which is expecting a statement, not NULL.

Error happens here: pegen.c:164
_PyPegen_get_expr_name(expr_ty e) is being called with e = 0x0

[miss-islington]

New changeset 9f49590 by Pablo Galindo in branch 'master':
bpo-40903: Handle multiple '=' in invalid assignment rules in the PEG parser (GH-20697)
9f49590

[miss-islington]

New changeset 8df4f39 by Miss Islington (bot) in branch '3.9':
bpo-40903: Handle multiple '=' in invalid assignment rules in the PEG parser (GH-20697)
8df4f39

[vstinner]

https://buildbot.python.org/all/#builders/765/builds/51

The new test fails with the old parser:

**********************************************************************
File "/buildbot/buildarea/3.9.pablogsal-arch-x86_64.oldparser/build/Lib/test/test_syntax.py", line 66, in test.test_syntax
Failed example:
    yield = 1
Expected:
    Traceback (most recent call last):
    SyntaxError: assignment to yield expression not possible
Got:
    Traceback (most recent call last):
      File "/buildbot/buildarea/3.9.pablogsal-arch-x86_64.oldparser/build/Lib/doctest.py", line 1336, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest test.test_syntax[10]>", line 1
        yield = 1
              ^
    SyntaxError: invalid syntax
**********************************************************************
1 items had failures:
   1 of 108 in test.test_syntax

[vstinner]

The new test fails with the old parser:

Oh, it's already fixed by:
2b33cc3