Support showing proc macro generated code?

[Timmmm]

I've been working on some wasmtime code and that makes extensive use of proc macros to generate Rust code.

It means you can end up writing code like this:

    wasmtime_wasi::p2::bindings::clocks::wall_clock::add_to_linker_get_host(l, closure)?;

You ctrl-click add_to_linker_get_host() to read the code, and it takes you here:

#[proc_macro]
pub fn bindgen(input: proc_macro::TokenStream) -> proc_macro::TokenStream {
    bindgen::expand(&parse_macro_input!(input as bindgen::Config))
        .unwrap_or_else(Error::into_compile_error)
        .into()
}

That... sucks. I think Rust-analyzer could do better though because if you hover add_to_linker_get_host it clearly knows about it:

This is maybe a better example. Hovering gives you great info about this type:

And you even get doc strings for members:

But if you ctrl-click the type you jump to exactly the same useless definition:

#[proc_macro]
pub fn bindgen(input: proc_macro::TokenStream) -> proc_macro::TokenStream {
    bindgen::expand(&parse_macro_input!(input as bindgen::Config))
        .unwrap_or_else(Error::into_compile_error)
        .into()
}

Would it be possible when you go-to-definition on these types to open an ephemeral editor and fill it with the generated code that the macro has written?

(Sorry if I'm not the first person to ask about this; I did try and search for duplicates but couldn't find any.)

[ChayimFriedman2]

This is exactly the goal of #19130. As you can see by the size of that PR, it isn't trivial.

[Timmmm]

Ah amazing! And wow you're right that is quite the PR!

[ChayimFriedman2]

Also note, I don't know how exactly your code is generated, but it jumping into the macro definition suggests the span are used incorrectly. If you use spans correctly, it should point to the point where the macro is called that creates the item, which while is not as helpful as going to the generated code is still better than going to the macro definition.

[Timmmm]

Ah interesting. I didn't write this code though unfortunately and have never written a proc macro. I think this function is responsible. It does seem to return Span::call_site() for errors, but maybe not for the generated code? I can't see how you do that tbf.

[ChayimFriedman2]

Span::call_site() for errors is the correct call. AFAIK what r-a is concerned with is mostly the name token, that is e.g. if you emit a struct Foo the Foo needs to have span written by the user.

[Timmmm]

Sorry, this is getting a bit off-topic but it would be great to improve this in wasmtime anyway. I couldn't really find any documentation for this stuff so I asked ChatGPT and it suggests that the tokens in the generated TokenStream can have a source Span associated with them, and that r-a will go to that span if you go-to-definition on a bit of code you have that uses that item (sounds reasonable).

Since the wasmtime proc macro generates code by reading a load of .wit files, could I set the source span for e.g. cli_exit_with_code to the .wit file it comes from? That would be really neat.

I guess it also raises a question wrt the actual topic here - if you have a item generated from a source file and you go-to-definition, do you want to see the generated Rust, or the original source? Probably you want the option of both right?

[Timmmm]

Actually, looking at this particular proc macro it unfortunately generates a big string and then turns that into a token stream, so annotating it properly would be difficult... but I did notice they have hacked a way of making this not so awful:

    // If a magical `WASMTIME_DEBUG_BINDGEN` environment variable is set then
    // place a formatted version of the expanded code into a file. This file
    // will then show up in rustc error messages for any codegen issues and can
    // be inspected manually.

[ChayimFriedman2]

Oh, they are parsing external files. So yes, it is harder. I don't think there is a way to point spans to an external file, unfortunately.

[Timmmm]

Ah yeah and I don't think WASMTIME_DEBUG_BINDGEN helps r-a in this scenario either, because it just allows writing the code to an external .rs file; you still can't use Span to point r-a to that file. There's an issue for that here.