Expect , for parenthesized with items and not at )
#10910
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dhruvmanila
added
bug
|
| code | total | + violation | - violation | + fix | - fix |
|---|---|---|---|---|---|
| PLR6104 | 92 | 0 | 92 | 0 | 0 |
| RUF100 | 26 | 26 | 0 | 0 | 0 |
| PLR1730 | 7 | 7 | 0 | 0 | 0 |
Formatter (stable)
✅ ecosystem check detected no format changes.
Formatter (preview)
✅ ecosystem check detected no format changes.
dhruvmanila
force-pushed
the
dhruv/with-item-expect-comma
branch
from
8f8c388
to
b953da8
Compare
dhruvmanila
commented
Apr 12, 2024
Comment on lines
+2070
to
+2078
| if with_item_kind == WithItemKind::Parenthesized && !self.at(TokenKind::Rpar) { | ||
| // test_err with_items_parenthesized_missing_comma | ||
| // with (item1 item2): ... | ||
| // with (item1 as f1 item2): ... | ||
| // with (item1, item2 item3, item4): ... | ||
| // with (item1, item2 as f1 item3, item4): ... | ||
| // with (item1, item2: ... | ||
| self.expect(TokenKind::Comma); | ||
| } |
There was a problem hiding this comment.
This is only the case for the first "gap" as any other missing commas will be highlighted by the list parsing in parse_with_items.
MichaReiser
approved these changes
Apr 15, 2024
dhruvmanila
force-pushed
the
dhruv/parser
branch
from
cefa753
to
509be82
Compare
dhruvmanila
force-pushed
the
dhruv/with-item-expect-comma
branch
from
b953da8
to
faf94e2
Compare
dhruvmanila
added a commit
that referenced
this pull request
Apr 16, 2024
## Summary This PR fixes a bug in parenthesized with-items parsing where the parser should expect a comma but it didn't. This is the case in the speculative parsing loop of parenthesized with-items. Once the parser finds any token other than comma, it breaks out of the loop. Then, once the parser has determined the with item kind, it needs to expect a comma if it's not the end of parenthesized with-items. ## Test Plan Add inline test cases, verify the snapshots.
dhruvmanila
added a commit
that referenced
this pull request
Apr 16, 2024
## Summary This PR fixes a bug in parenthesized with-items parsing where the parser should expect a comma but it didn't. This is the case in the speculative parsing loop of parenthesized with-items. Once the parser finds any token other than comma, it breaks out of the loop. Then, once the parser has determined the with item kind, it needs to expect a comma if it's not the end of parenthesized with-items. ## Test Plan Add inline test cases, verify the snapshots.
dhruvmanila
added a commit
that referenced
this pull request
Apr 17, 2024
## Summary This PR fixes a bug in parenthesized with-items parsing where the parser should expect a comma but it didn't. This is the case in the speculative parsing loop of parenthesized with-items. Once the parser finds any token other than comma, it breaks out of the loop. Then, once the parser has determined the with item kind, it needs to expect a comma if it's not the end of parenthesized with-items. ## Test Plan Add inline test cases, verify the snapshots.
dhruvmanila
added a commit
that referenced
this pull request
Apr 18, 2024
## Summary This PR fixes a bug in parenthesized with-items parsing where the parser should expect a comma but it didn't. This is the case in the speculative parsing loop of parenthesized with-items. Once the parser finds any token other than comma, it breaks out of the loop. Then, once the parser has determined the with item kind, it needs to expect a comma if it's not the end of parenthesized with-items. ## Test Plan Add inline test cases, verify the snapshots.
dhruvmanila
added a commit
that referenced
this pull request
Apr 18, 2024
(Supersedes #9152, authored by @LaBatata101) ## Summary This PR replaces the current parser generated from LALRPOP to a hand-written recursive descent parser. It also updates the grammar for [PEP 646](https://peps.python.org/pep-0646/) so that the parser outputs the correct AST. For example, in `data[*x]`, the index expression is now a tuple with a single starred expression instead of just a starred expression. Beyond the performance improvements, the parser is also error resilient and can provide better error messages. The behavior as seen by any downstream tools isn't changed. That is, the linter and formatter can still assume that the parser will _stop_ at the first syntax error. This will be updated in the following months. For more details about the change here, refer to the PR corresponding to the individual commits and the release blog post. ## Test Plan Write _lots_ and _lots_ of tests for both valid and invalid syntax and verify the output. ## Acknowledgements - @MichaReiser for reviewing 100+ parser PRs and continuously providing guidance throughout the project - @LaBatata101 for initiating the transition to a hand-written parser in #9152 - @addisoncrump for implementing the fuzzer which helped [catch](#10903) [a](#10910) [lot](#10966) [of](#10896) [bugs](#10877) --------- Co-authored-by: Victor Hugo Gomes <labatata101@linuxmail.org> Co-authored-by: Micha Reiser <micha@reiser.io>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes a bug in parenthesized with-items parsing where the parser should expect a comma but it didn't.
This is the case in the speculative parsing loop of parenthesized with-items. Once the parser finds any token other than comma, it breaks out of the loop. Then, once the parser has determined the with item kind, it needs to expect a comma if it's not the end of parenthesized with-items.
Test Plan
Add inline test cases, verify the snapshots.