Implement raw lifetimes and labels ('r#ident)

[compiler-errors]

This PR implements a syntax for raw lifetimes and labels as requested in #126335.

This PR also moves keyword lifetime validation (i.e. checking that we don't have something like 'async) from ast validation to the parser, similar to how we handle raw identifiers, since we don't distinguish raw identifiers after parsing.

This PR additionally extends the keyword_idents_2024 lint to also check identifiers.

cc @traviscross
r? parser

[compiler-errors]

The only theoretical breakage is this:

macro_rules! lt {
    ($lt:lifetime # $id:ident) => {}
}

lt!('r#a);

edit: whoops, @fmease also notes that it breaks:

#[cfg(any())] fn f<'async>(_: &'async ()) {}

[compiler-errors]

Also, this doesn't reserve all prefixed lifetimes -- it just implements 'r#. Not sure if T-lang wants all prefixes to be reserved as well.

[rustbot]

rust-analyzer is developed in its own repository. If possible, consider making this change to rust-lang/rust-analyzer instead.

cc @rust-lang/rust-analyzer

Some changes occurred in src/tools/rustfmt

cc @rust-lang/rustfmt

[compiler-errors]

r-a and rustfmt changes are required due to changes to lexer and parser, respectively.

[ytmimi]

rustfmt changes look straightforward to me. Do you think it's worth adding a basic test in src/tools/rustfmt/tests/target for these new raw lifetimes and labels?

[compiler-errors]

Oh good point. I have no idea if raw lifetimes even format correctly 😅

[ytmimi]

🤞🏼fingers crossed they just work out of the box. Given the changes to the parser does the span include the whole 'r#label or 'r#lifetime?

[compiler-errors]

should include the whole span, so if we extract the lifetime name from the span (and not from the ident.name Symbol) then we should be fine 👍 I'll implement those tests tho

[rust-log-analyzer]

The job x86_64-gnu-tools failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

  |

........
Mismatch at src/macros.rs:1097:
         | TokenKind::OpenDelim(_)
         | TokenKind::CloseDelim(_) => SpaceState::Never,
 
-        TokenKind::Literal(..) | TokenKind::Ident(..) | TokenKind::Lifetime(..) => SpaceState::Ident,
+        TokenKind::Literal(..) | TokenKind::Ident(..) | TokenKind::Lifetime(..) => {
+        }
 
         _ => SpaceState::Always,
     }
     }
{ "type": "test", "name": "test::self_tests", "event": "failed", "stdout": "Ran 5 self tests.\nthread 'test::self_tests' panicked at src/tools/rustfmt/src/test/mod.rs:400:5:\nassertion `left == right` failed: 1 self tests failed\n  left: 1\n right: 0\nnote: run with `RUST_BACKTRACE=1` environment variable to display a backtrace\n" }

test result: FAILED. 173 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 984.12ms

error: test failed, to rerun pass `--lib`

[fmease]

This PR also moves keyword lifetime validation (i.e. checking that we don't have something like 'async) from ast validation to the parser

For the T-lang meeting: Theoretical breakage of stable code:

#[cfg(any())] fn f<'async>(_: &'async ()) {}

[compiler-errors]

@bors try

[bors]

⌛ Trying commit 0129478 with merge d9cbfeb...

[bors]

☀️ Try build successful - checks-actions
Build commit: d9cbfeb (d9cbfeb054d7bd02cf942e6d32c1c0a91f042ce9)

[compiler-errors]

@craterbot check

[craterbot]

👌 Experiment pr-126452 created and queued.
🤖 Automatically detected try build d9cbfeb
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[craterbot]

🚧 Experiment pr-126452 is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[craterbot]

🎉 Experiment pr-126452 is completed!
📊 11 regressed and 2 fixed (472186 total)
📰 Open the full report.

⚠️ If you notice any spurious failure please add them to the blacklist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[compiler-errors]

There's a crate that specifically abuses 'ref which is a keyword lifetime. Might have to find a workaround for this.

[danielhenrymantilla]

FWIW, as documented over https://docs.rs/with_locals/0.3.2/with_locals/index.html#remarks, the 'ref lifetime choice is just a default one. Users can always provide their own lifetime marker instead. This means that it is possible for users to adjust their code to the change of this PR if needs be.

• (if we could afford the wait, I could even patch the crate to start warning about the direct 'ref usage (DIY FCW, sort of))

[traviscross]

@rustbot labels -I-lang-nominated

We discussed this in the lang triage call today.

We confirmed that we mean for all prefixed lifetimes to be reserved, not just 'r# ones. We view both the missing reservation and the missing support for 'r# as bug fixes consistent with the intent of RFC 3101. The result of this reservation is that these prefixed lifetimes will be parsed as single tokens rather than as many.

Secondly, we confirmed that we're OK with moving the validation of keywords in lifetimes to pre-expansion from post-expansion. We similarly consider this a bug fix. While the breakage of the convenience feature of the with_locals crate that relies on this is unfortunate, and we wish we had not overlooked this earlier for that reason, we're fortunate that the breakage is contained to only one crate, and we're going to accept this breakage as the extra complexity we'd need to carry in the compiler to work around this isn't deemed worth it.

Thanks in particular to @danielhenrymantilla for being responsive on this and understanding of the situation.