refactor expr & stmt parsing + improve recovery #66994

Centril · 2019-12-03T18:03:55Z

Summary of important changes (best read commit-by-commit, ignoring whitespace changes):

AttrVec is introduces as an alias for ThinVec<Attribute>
parse_expr_bottom and parse_stmt are thoroughly refactored.
Extract diagnostics logic for vec![...] in a pattern context.
Recovery is added for do catch { ... }
Recovery is added for 'label: non_block_expr
Recovery is added for var $local, auto $local, and mut $local. Fixes Use 'mut' as alias for 'let mut' #65257.
Recovery is added for e1 and e2 and e1 or e2.
~~macro_legacy_warnings is turned into an error (has been a warning for 3 years!)~~
Fixes Clean up parse_bottom_expr to use list parsing utility #63396 by forward-porting Clean up parse_bottom_expr to use list parsing utility #64105 which now works thanks to added recovery.
ui-fulldeps/ast_stmt_expr_attr.rs is turned into UI and pretty tests.
Recovery is fixed for #[attr] if expr {}

Centril · 2019-12-03T19:28:21Z

Looking at some more changes to expr.rs right now because it's still a mess. :(
Will make those changes as a different PR based on this one tho.

estebank · 2019-12-04T00:15:52Z

src/test/ui/parser/stmt_expr_attrs_placement.stderr

+LL |     let a = #![allow(warnings)] (1, 2);
+   |             ^^^^^^^^^^^^^^^^^^^
+   |
+   = note: inner attributes, like `#![no_std]`, annotate the item enclosing them, and are usually found at the beginning of source files. Outer attributes, like `#[test]`, annotate the item following them.


We don't have a style guide covering this yet, but this mix&match of lowercase led sentence followed by properly capitalized sentence is jarring to me. In other errors I've sidestepped the issue by issuing two notes or reworking the text to work as a single sentence.

Well it's pre-existing and I would prefer not to do anything about it here, but maybe you want to file an issue for this or something. :)

estebank · 2019-12-04T00:16:18Z

src/test/ui/parser/stmt_expr_attrs_placement.rs

+    let a = #![allow(warnings)] (1, 2);
+    //~^ ERROR an inner attribute is not permitted in this context
+
+    let b = (#![allow(warnings)] 1, 2);


These other cases should also be emitting warnings, right?

Well warnings are being allowed so that doesn't seem right. The point of this test is only parsing behavior tho, not the lint system.

src/test/ui/parser/recover-labeled-non-block-expr.stderr

src/test/ui/did_you_mean/issue-54109-and_instead_of_ampersands.stderr

estebank · 2019-12-04T00:21:24Z

src/test/ui/did_you_mean/issue-54109-and_instead_of_ampersands.stderr

+  --> $DIR/issue-54109-and_instead_of_ampersands.rs:20:15
+   |
+LL |     let _ = a or b;
+   |               ^^ help: instead of `or`, use `||` to perform logical disjunction: `||`


"logical conjuction/disjunction" makes sense and is accurate, but it's not very friendly to most people, who will most likely need to do a double take to understand what it means (if we reword in a more terse manner as suggested above).

I'm not sure what a less jargon-y wording would be. At least "logical conjunction" is easy to google for.

src/test/ui/parser/issue-65257-invalid-var-decl-recovery.stderr

estebank · 2019-12-04T00:27:58Z

src/librustc_parse/parser/stmt.rs

@@ -426,11 +410,11 @@ impl<'a> Parser<'a> {
    }

    /// Parses a statement, including the trailing semicolon.
-    pub fn parse_full_stmt(&mut self, macro_legacy_warnings: bool) -> PResult<'a, Option<Stmt>> {
+    pub fn parse_full_stmt(&mut self) -> PResult<'a, Option<Stmt>> {


@petrochenkov you'll want to take a look at the changes in this file.

src/librustc_parse/parser/expr.rs

estebank · 2019-12-04T00:47:15Z

src/librustc_parse/parser/expr.rs

+            .span_label(self.token.span, msg)
+            .emit();
+        // Continue as an expression in an effort to recover on `'label: non_block_expr`.
+        self.parse_expr()


Should this use the snapshotting hack to to try and keep the error count down? There are cases where error recovery is too eager and we end up with wall of texts on a single typo :-/

To be clear, this is fine as is, just something to consider when working on this code doing these kind of things. If an expression can't be parsed, at that point it is nicer to just give up and raise FatalError.

Imo not worth it until more people complain :)

What if I'm the one complaining about the output I'm seeing? :^)

Are you tho? ;)

estebank · 2019-12-04T05:58:29Z

src/librustc_parse/parser/stmt.rs

-            &format!("expected `;`, found {}", self.this_token_descr())
-        }).note({
-            "this was erroneously allowed and will become a hard error in a future release"
-        }).emit();


Where did this go? We can't suddenly start rejecting code, even if we were unconditionally warning on it.

It was always intended to become an error eventually and has been a warning for 3 years, so I figured we can turn it into a hard error by now.

SGTM, make sure to run crater :)

I'll remove it from the PR for now so we don't have to wait for crater and so I don't have to rebase. I've filed #67098 as a reminder.

estebank

LGTM, besides my prior comments

Centril · 2019-12-06T19:35:50Z

Rebased & dropped the legacy warning stuff for a future PR.

Centril · 2019-12-20T22:08:34Z

Rebased. :)

estebank · 2019-12-21T02:24:31Z

src/test/ui/did_you_mean/issue-54109-and_instead_of_ampersands.stderr

+LL |     let _ = a and b;
+   |               ^^^ help: use `&&` to perform logical conjunction
+   |
+   = note: unlike in e.g., python and PHP, `&&` and `||` are used for logical operators


I don't know if the note calling out python and PHP here is a good idea.

I debated this with myself a few times but in the end I thought that habits from python (PHP less so) would be the most likely source of using and so it felt like relevant information for such users.

estebank · 2019-12-21T02:29:32Z

@bors r+

bors · 2019-12-21T02:29:33Z

📌 Commit 621661f has been approved by estebank

bors · 2019-12-21T02:29:34Z

🌲 The tree is currently closed for pull requests below priority 100, this pull request will be tested once the tree is reopened

Centril · 2019-12-21T11:04:57Z

@bors p=199

bors · 2019-12-21T11:05:08Z

⌛ Testing commit 621661f with merge c64eecf...

@estebank

refactor expr & stmt parsing + improve recovery Summary of important changes (best read commit-by-commit, ignoring whitespace changes): - `AttrVec` is introduces as an alias for `ThinVec<Attribute>` - `parse_expr_bottom` and `parse_stmt` are thoroughly refactored. - Extract diagnostics logic for `vec![...]` in a pattern context. - Recovery is added for `do catch { ... }` - Recovery is added for `'label: non_block_expr` - Recovery is added for `var $local`, `auto $local`, and `mut $local`. Fixes #65257. - Recovery is added for `e1 and e2` and `e1 or e2`. - ~~`macro_legacy_warnings` is turned into an error (has been a warning for 3 years!)~~ - Fixes #63396 by forward-porting #64105 which now works thanks to added recovery. - `ui-fulldeps/ast_stmt_expr_attr.rs` is turned into UI and pretty tests. - Recovery is fixed for `#[attr] if expr {}` r? @estebank

bors · 2019-12-21T14:21:49Z

☀️ Test successful - checks-azure
Approved by: estebank
Pushing c64eecf to master...

@estebank

Refactor expression parsing thoroughly Based on rust-lang#66994 together with which this has refactored basically the entirety of `expr.rs`. r? @estebank

@estebank

Refactor expression parsing thoroughly Based on #66994 together with which this has refactored basically the entirety of `expr.rs`. r? @estebank

@estebank

Refactor expression parsing thoroughly Based on #66994 together with which this has refactored basically the entirety of `expr.rs`. r? @estebank

rust-highfive assigned estebank Dec 3, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Dec 3, 2019

Centril force-pushed the stmt-polish branch from 90a3771 to a5ab384 Compare December 3, 2019 18:20

petrochenkov self-assigned this Dec 3, 2019

This comment has been minimized.

Sign in to view