Skip to content

Python: fix match-statement parse desync that corrupts trees over RPC#8330

Open
shanman190 wants to merge 2 commits into
mainfrom
python-match-parser-desync
Open

Python: fix match-statement parse desync that corrupts trees over RPC#8330
shanman190 wants to merge 2 commits into
mainfrom
python-match-parser-desync

Conversation

@shanman190

Copy link
Copy Markdown
Contributor

Two spots in the Python match-case parser left source unconsumed, drifting the source cursor so every later node parsed against misaligned text and came out with None children. RpcSendQueue then crashes at el.element._id on the None child, failing the tree transfer (handle_get_object returns a degenerate [END_OF_OBJECT]). Because match/case is Python 3.10+ and absent from older codebases, the defect only surfaces on modernized code.

Fixes (both in _parser_visitor.py)

  • Guards on value/capture patterns (case 1 if x:, case Enum.X if x:, case v if x:): the guard source (if <cond>) was consumed only when the pattern wrapped as py.MatchCase. Value and capture patterns are returned bare, so the guard text stayed in the buffer. Consume it unconditionally and wrap the bare pattern in py.MatchCase(VALUE/CAPTURE) so the guard has a home. The existing py.MatchCase codec (Python and Java) and printer already handle these kinds — no model or codec change needed.
  • Parenthesized GROUP as a class keyword value (case Foo(k=(A() | B())):): keyword patterns used __convert (which does not consume parens) while positional patterns used __convert_match_pattern. Use the latter for both, consistent with positional handling.

Verification

  • New regression tests (tests/rpc/test_match_guard_desync.py) cover both shapes end-to-end (parse → RpcSendQueueRpcReceiveQueue → print), asserting no None-element padding and a clean round-trip.
  • Serialize→deserialize→print round-trip run over 20,635 real files (numpy, home-assistant, ansible, botocore): 3 crashing files → 0, no regressions, existing match/pattern suite green.

Two spots in the match-case parser left source unconsumed, drifting the cursor
so every later node parsed against misaligned text and came out with None
children. RpcSendQueue then crashes at `el.element._id` on the None child,
failing the tree transfer (handle_get_object returns a degenerate
[END_OF_OBJECT]). match/case is 3.10+ and absent from older corpora, so the
defect only surfaced on modernized code.

- Guards on value/capture patterns: the guard source (`if <cond>`) was consumed
  only when the pattern wrapped as py.MatchCase. Value and capture patterns are
  returned bare, so the guard text stayed in the buffer. Consume it
  unconditionally and wrap the bare pattern in py.MatchCase(VALUE/CAPTURE) so the
  guard has a home; the existing py.MatchCase codec (Python and Java) and printer
  already handle these kinds.
- Parenthesized GROUP as a class keyword value (`case Foo(k=(A() | B())):`):
  keyword patterns used __convert, which doesn't consume parens, while positional
  patterns used __convert_match_pattern. Use the latter for both.

Verified across 20,635 files (numpy, home-assistant, ansible, botocore):
3 crashes -> 0, no regressions.
Comment thread rewrite-python/rewrite/src/rewrite/python/_parser_visitor.py Outdated
Comment thread rewrite-python/rewrite/src/rewrite/python/_parser_visitor.py Outdated
Co-authored-by: Shannon Pamperl <shanman190@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

1 participant