Implement literal_from_str for proc macro server

[Tyrubias]

Closes #16233

Todos and unanswered questions:

• Is this the correct approach? Can both the legacy and rust_analyzer_span servers depend on the syntax crate?
• How should we handle suffixes for string literals? It doesn't seem like rust-analyzer preservers suffix information after parsing.
• Why are the expect tests failing? Specifically test_fn_like_macro_clone_literals

[Veykril]

Can both the legacy and rust_analyzer_span servers depend on the syntax crate?

Yes thats fine, the server depends on tt which depends on syntax and parser

How should we handle suffixes for string literals? It doesn't seem like rust-analyzer preservers suffix information after parsing.

What do those look like? I would expect r-a to handle them the same as with integer suffixes

[Tyrubias]

In regards to the suffixes for strings, the Rust Reference has this to say:

Any kind of literal (string, integer, etc) with any suffix is valid as a token.

A literal token with any suffix can be passed to a macro without producing an error. The macro itself will decide how to interpret such a token and whether to produce an error or not.

My understanding is we will have to handle literals like

let item = "crab"cake;

Currently in syntax::ast::token_ext we have logic which iterates over integers and floats and identifies and extracts the suffixes. Should we implement similar logic for anything implementing syntax::ast::token_ext::IsString? Or should there be a bigger effort to extract suffixes during the parsing stage instead?

In regards to the other comments, I will address them and re-request a review.

Thanks for the feedback!

[Veykril]

Should we implement similar logic for anything implementing syntax::ast::token_ext::IsString? Or should there be a bigger effort to extract suffixes during the parsing stage instead?

We should go about this the same way we handle it for integers and such. The parser currently intentionally does not acknowledge suffixes separately (as they are technically part of the token).

[Tyrubias]

I think I might open a separate PR to deal with parsing the suffix for strings. Is that ok with you?

[Veykril]

Thanks!
@bors r+

(gonna remove some allocations in that code in a follow up in a bit)

[bors]

📌 Commit 4923b8a has been approved by Veykril

It is now in the queue for this repository.

[bors]

⌛ Testing commit 4923b8a with merge 925705e...

[bors]

☀️ Test successful - checks-actions
Approved by: Veykril
Pushing 925705e to master...