Fix, document, and test parser and pretty-printer edge cases related to braced macro calls

[dtolnay]

Review note: this is a deceptively small PR because it comes with 145 lines of docs and 196 lines of tests, and only 25 lines of compiler code changed. However, I recommend reviewing it 1 commit at a time because much of the effect of the code changes is non-local i.e. affecting code that is not visible in the final state of the PR. I have paid attention that reviewing the PR one commit at a time is as easy as I can make it. All of the code you need to know about is touched in those commits, even if some of those changes disappear by the end of the stack.

This is a follow-up to #119105. One case that is not relevant to -Zunpretty=expanded, but which came up as I'm porting #119105 and #118726 into syn's printer and prettyplease's printer where it is relevant, and is also relevant to rustc's stringify!, is statement boundaries in the vicinity of braced macro calls.

Rustc's AST pretty-printer produces invalid syntax for statements that begin with a braced macro call:

macro_rules! stringify_item {
    ($i:item) => {
        stringify!($i)
    };
}

macro_rules! repro {
    ($e:expr) => {
        stringify_item!(fn main() { $e + 1; })
    };
}

fn main() {
    println!("{}", repro!(m! {}));
}

Before this PR: output is not valid Rust syntax.

fn main() { m! {} + 1; }

error: leading `+` is not supported
 --> <anon>:1:19
  |
1 | fn main() { m! {} + 1; }
  |                   ^ unexpected `+`
  |
help: try removing the `+`
  |
1 - fn main() { m! {} + 1; }
1 + fn main() { m! {}  1; }
  |

After this PR: valid syntax.

fn main() { (m! {}) + 1; }

[rustbot]

r? @WaffleLapkin

(rustbot has picked a reviewer for you, use r? to override)

[WaffleLapkin]

I think I agree with the idea, but I'm not sure about the implementation. Is there a way to make this nicer?

[WaffleLapkin]

why is this needed, if all the call sites are not calling this if e ≈ MacCall?

[dtolnay]

In the commit that adds this MacCall case inside expr_requires_semi_to_be_stmt, I change all the call sites to preserve their pre-existing behavior. So this case starts out as not reachable. But after that, one at a time for each call site, I update each call site that should be reaching this, together with adding documentation and tests for that call site.

[bors]

☔ The latest upstream changes (presumably #120121) made this pull request unmergeable. Please resolve the merge conflicts.

[WaffleLapkin]

r? compiler

[fmease]

#120690
r? compiler

[dtolnay]

This is waiting on me to incorporate Waffle's insightful feedback, and to rebase.

[dtolnay]

@rustbot ready

[michaelwoerister]

Thanks for the PR, @dtolnay, looks great! However, after looking through all the commits, I think I don't understand enough about the parsing subtleties involved here to review this.

r? parser

[dtolnay]

• Rebased to fix conflict with Stabilise inline_const #104087

[compiler-errors]

This is maybe my only nit in the PR: that expr_is_complete uses expr_requires_comma_to_be_match_arm "feels" a bit less self-descriptive than it could be, since we use expr_is_complete for things like parse_expr_dot_or_call_with_ which doesn't really have to do with match arms.

Not really sure how to make this actionable though, since I'm not sure if a rename seems tough. AFAICT this just preserves the old behavior, right?

[dtolnay]

Right, this is preserving old behavior. The exact same behavior that used to be incorrectly called expr_requires_semi_to_be_stmt is now called expr_requires_comma_to_be_match_arm.

Regarding naming, one possibility is that instead of distinguishing a "commas in match arms" and "semicolons in statements" case (and the implicit 3rd case, expressions that are neither statement nor match arm, such as a function arg), we could rename things along the following lines:

• "parse the longest possible expression"
• "parse an expression using earlier boundary rule" (for a match arm, range, or whatever else looks at expr_is_complete)
• "parse an expression until the earliest possible boundary" (for a statement)

Another possibility is to omit this specific commit ("Add classify::expr_requires_comma_to_be_match_arm") and keep using match e { MacCall(_) => …, _ => expr_requires_semi_to_be_stmt(e) } in the cases where expr_requires_semi_to_be_stmt is not the desired behavior.

[dtolnay]

One more option — instead of having what's currently in the PR:

// compiler/rustc_ast/src/util/classify.rs

pub fn expr_requires_semi_to_be_stmt(e: &ast::Expr) -> bool {
    match &e.kind {
        If(..)
        | Match(..)
        | Block(..)
        | While(..)
        | Loop(..)
        | ForLoop { .. }
        | TryBlock(..)
        | ConstBlock(..) => false,

        MacCall(mac_call) => mac_call.args.delim != Delimiter::Brace,

        _ => true,
    }
}

pub fn expr_requires_comma_to_be_match_arm(e: &ast::Expr) -> bool {
    match &e.kind {
        MacCall(_) => true,
        _ => expr_requires_semi_to_be_stmt(e),
    }
}

we could have:

pub fn expr_is_complete(e: &ast::Expr) -> bool {
    // this is negative of the old "expr_requires_semi_to_be_stmt"
    match &e.kind {
        If(..)
        | Match(..)
        | Block(..)
        | While(..)
        | Loop(..)
        | ForLoop { .. }
        | TryBlock(..)
        | ConstBlock(..) => true,

        MacCall(_) | _ => false,
    }
}

pub fn expr_requires_semi_to_be_stmt(e: &ast::Expr) -> bool {
    match &e.kind {
        MacCall(mac_call) => mac_call.args.delim != Delimiter::Brace,
        _ => !expr_is_complete(e),
    }
}

[compiler-errors]

I prefer either the final option or the renaming you suggested first above. I'll leave it up to you. Otherwise this PR looks good to me.

[dtolnay]

I removed 5d5954c and added 10227ea.

[compiler-errors]

🆒

[compiler-errors]

@bors r+

[bors]

📌 Commit a9ec581 has been approved by compiler-errors

It is now in the queue for this repository.

[dtolnay]

@bors r=compiler-errors

[bors]

📌 Commit 78c8dc1 has been approved by compiler-errors

It is now in the queue for this repository.

[bors]

⌛ Testing commit 78c8dc1 with merge 8cc6f34...

[bors]

☀️ Test successful - checks-actions
Approved by: compiler-errors
Pushing 8cc6f34 to master...

[rust-timer]

Finished benchmarking commit (8cc6f34): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	4.1%	[4.1%, 4.1%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-2.5%	[-2.5%, -2.5%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.8%	[-2.5%, 4.1%]	2

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	3.8%	[2.8%, 5.6%]	3
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 675.636s -> 678.17s (0.38%)
Artifact size: 315.95 MiB -> 315.89 MiB (-0.02%)