{{ message }}
/ rust Public

# Attribute macros invoked at crate root have issues#41430

Open
opened this issue Apr 20, 2017 · 29 comments
Open

# Attribute macros invoked at crate root have issues #41430

opened this issue Apr 20, 2017 · 29 comments
Labels
A-attributes A-macros-2.0 C-bug T-compiler

### abonander commented Apr 20, 2017 • edited

 When invoking some attribute procedural macro with inner form, i.e. #![foo_attr] at the crate root, a resolution error occurs: #![feature(proc_macro)] #![foo_attr] //^ ERROR: cannot find attribute macro foo_attr in this scope extern crate foo_macros; use foo_attr_macro::foo_attr; This should, ideally, resolve and execute properly. The text was updated successfully, but these errors were encountered:

### jseyfried commented Apr 20, 2017 • edited

 @abonander What about this? #![feature(proc_macro)] extern crate foo_macros; use foo_macros::foo_attr; mod bar { #![foo_attr] } If compiles, then the issue is that the attribute is at the crate root, not that it is an inner attribute.

### abonander commented Apr 20, 2017

 @jseyfried Updated issue as this only affects the crate root

added a commit to abonander/rust that referenced this issue Apr 20, 2017
 Don't panic if an attribute macro fails to resolve at crate root 
 910532e 
Adds temporary regression test; this ideally should work as-is (rust-lang#41430)

Closes rust-lang#41211
mentioned this issue Apr 20, 2017
changed the title Attribute macros invoked with inner-attribute form fail to resolve Attribute macros invoked at crate root fail to resolve Apr 21, 2017

### abonander commented Apr 21, 2017 • edited by jseyfried

 So @jseyfried and I figured out on IRC that this is due to the compiler trying to expand the attribute before the use_extern_macros code gets to see the actual import. I'd really like to make this work as I have some cool use-cases for it. We had a couple ideas for how to get this working but we figured we should expand the discussion on it before taking any action. My best idea is along the lines of this: if an attribute macro fails to resolve at crate root only (since inner attributes on modules should resolve in the parent's scope), import-process root, get the resolution for the attribute, reinitialize the resolver and copy the resolution for the attribute macro, then expand the root module. Some concerns from @jseyfried: I believe that would be doable implementation-wise, but it could lead to coherence issues For example, if the attribute macro expanded into nothing Or if it expanded into a different import of itself The resolution of a name isn't supposed to change as we expand things It can only go from "indeterminate" to "success" or "indeterminate" to "failed" If we preserve that with extra checks, what you're proposing could work though, I think cc @nrc

added a commit to frewsxcv/rust that referenced this issue Apr 22, 2017
 Rollup merge of rust-lang#41432 - abonander:issue_41211, r=jseyfried 
 c150d0e 
Don't panic if an attribute macro fails to resolve at crate root

Adds temporary regression test; this ideally should work as-is (rust-lang#41430)

Closes rust-lang#41211

r? @jseyfried
added a commit to frewsxcv/rust that referenced this issue Apr 22, 2017
 Rollup merge of rust-lang#41432 - abonander:issue_41211, r=jseyfried 
 9283402 
Don't panic if an attribute macro fails to resolve at crate root

Adds temporary regression test; this ideally should work as-is (rust-lang#41430)

Closes rust-lang#41211

r? @jseyfried
added a commit to frewsxcv/rust that referenced this issue Apr 22, 2017
 Rollup merge of rust-lang#41432 - abonander:issue_41211, r=jseyfried 
 48a9d5f 
Don't panic if an attribute macro fails to resolve at crate root

Adds temporary regression test; this ideally should work as-is (rust-lang#41430)

Closes rust-lang#41211

r? @jseyfried

### abonander commented May 15, 2017

 @nrc Some input here when you get the chance, I'd like to tackle this soonish.

### nrc commented May 16, 2017

 sorry it took a while to see this - I was away on parental leave and then had to nuke my notifications. since inner attributes on modules should resolve in the parent's scope I wonder if this axiom is correct? An alternative seems to be that we could resolve attributes at the scope in which they are written rather than the scope in which they apply. Would that have knock-on effects?

### jseyfried commented May 16, 2017 • edited

 resolve attributes at the scope in which they are written rather than the scope in which they apply Interesting, I like this idea. I'm not aware of any knock-on effects, it should be straightforward to implement, and it would address this use case.

### abonander commented May 16, 2017

 I was under the impression that inner attributes were syntactic sugar for outer attributes, and it makes sense for outer attributes to resolve in the same scope as the item they're applied to. Crates are just special because there's no outer scope to resolve.

### nrc commented May 17, 2017

 I was under the impression that inner attributes were syntactic sugar for outer attributes Whether a feature is desugared or not is an implementation detail and we should do what makes the most sense from the user's POV, not the compilers. Desugaring can take place after name resolution if we like, so we don't need to change the implementation either.

### abonander commented May 20, 2017

 So the plan of action is to just reorder attribute expansion and import processing for a given item? Or just at the crate root for now?

### jseyfried commented May 20, 2017 • edited

 @abonander I wouldn't think of this as reordering attribute expansion or import processing, just changing the scope in which e.g. bar resolves in #[foo] mod m { #![bar] #[baz] fn f() {} ... } from that of foo to that of baz.

### jseyfried commented May 20, 2017

 If we make this change, I think we should make the change everywhere (not just at the crate root) for consistency.

mentioned this issue Jun 16, 2017
added the A-macros-2.0 label Jun 23, 2017
mentioned this issue Jun 30, 2017
added the C-bug label Jul 27, 2017

### Rantanen commented Dec 29, 2017

 #44660 improves the situation. The following compiles okay on nightly: #![feature(extern_absolute_paths, proc_macro)] #![::my_crate::my_attribute]  I believe the absoluteness of the path can be omitted in the future, once the RFC 2126 implementation proceeds further. However #45458 still makes using the crate-level macros difficult. The issue concerns compiler plugins, but can also be reproduced through attribute macros.

mentioned this issue Feb 7, 2018
mentioned this issue Feb 8, 2018

### Rantanen commented Feb 8, 2018 • edited

Moving from #48066 as per @Manishearth wishes. Although I would prefer if the title of this issue was updated - the resolution itself is somewhat "solved" in nightly. Even if the current ::absolute syntax ends up not being the final one, I'd expect the same mechanism to work with whichever syntax the module update ends up implementing.

While normal proc-macro attribute work, crate level proc-macro attributes seem to have several issues. The clearest of these is the order of the attributes and extern crate statements: #41430. #![feature(extern_absolute_paths)] feature can work around this specific issue; However this alone doesn't make proc macros work.

Some of these issues are due to lack of library support (syn), but some are caused by rustc itself.

## Identity macro ✔️

First of all: The following proc-macro works just fine with nightly 29c8276:

#[proc_macro_attribute]
fn attribute(_: TokenStream, input: TokenStream) -> TokenStream {
input
}
#![feature(extern_absolute_paths)]
#![::my_crate::attribute]
fn main() {}

## Round trip through FromIterator ❌

As long as the macro doesn't touch the input, everything works just fine. However this isn't really that useful for a macro. Once we try doing anything with the input token stream (short of a .clone()), things start breaking.

#[proc_macro_attribute]
fn attribute(_: TokenStream, input: TokenStream) -> TokenStream {
TokenStream::from_iter( input.into_iter() )
}
#![feature(extern_absolute_paths)]
#![::my_crate::attribute]
fn main() {}

This results in the following error message

error: expected identifier, found {
--> <macro expansion>:1:1
|
1 | pub mod  {
| ^^^ expected identifier


The reason seems to be that the TokenStream the attribute receives looks like:

pub mod {
...
}

This is most likely caused by the ast::Crate modeling the items as a Mod with no Ident: https://github.com/rust-lang/rust/blob/master/src/libsyntax/ast.rs#L447-L451

## Stripping the pub mod away ❌

Implementing a Crate type for Syn and having its to_tokens skip the pub mod { .. } bits allows the parser to process the output correctly. I'm also hoping this would be the way to go, since the parse_crate_mod method just parses inner attributes and mod items without seemingly caring about the mod keywords or braces.

However now we encounter the following ICE:

thread 'rustc' panicked at 'internal error: entered unreachable code', libsyntax/ext/expand.rs:258:18

error: an inner attribute is not permitted in this context
|
= note: inner attributes, like #![no_std], annotate the item enclosing them, and are usually found at the beginning of source files. Outer attributes, like #[test], annotate the item following them.

thread 'rustc' panicked at 'internal error: entered unreachable code', libsyntax/ext/expand.rs:258:18
note: Some details are omitted, run with RUST_BACKTRACE=full for a verbose backtrace.
stack backtrace:
0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
at libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
1: std::sys_common::backtrace::_print
at libstd/sys_common/backtrace.rs:71
2: std::panicking::default_hook::{{closure}}
at libstd/sys_common/backtrace.rs:59
at libstd/panicking.rs:380
3: std::panicking::default_hook
at libstd/panicking.rs:396
4: std::panicking::rust_panic_with_hook
at libstd/panicking.rs:576
5: std::panicking::begin_panic
6: syntax::ext::expand::MacroExpander::expand_crate
7: rustc_driver::driver::phase_2_configure_and_expand_inner::{{closure}}
8: rustc_driver::driver::phase_2_configure_and_expand_inner
9: rustc_driver::driver::compile_input
10: rustc_driver::run_compiler

error: internal compiler error: unexpected panic


As far as I can tell, the only way for this ICE to happen is for the MacroExpander to successfully expand the attribute. However the code expects the expansion to result in ast::Item(Mod)

I tried compiling a debug version of rustc for this, however then I end up with...

## syn::parse(input) panics ❌

At this point my attribute fn does nothing but parses and quotes the input using syn using a custom Crate type that handles pub mod without an ident:

#[proc_macro_attribute]
pub fn attribute(_attr: TokenStream, input: TokenStream) -> TokenStream {
let file : Crate = parse( input ).unwrap();
quote!( #file ).into()
}

This results in the following panic:

thread '' panicked at 'proc_macro::__internal::with_sess() called before set_parse_sess()!',

   Compiling test_crate v0.1.0 (file:///C:/Dev/Projects/crate_level_proc_macro/test_crate)
thread '<unnamed>' panicked at 'proc_macro::__internal::with_sess() called before set_parse_sess()!', libproc_macro\lib.rs:851:9
note: Some details are omitted, run with RUST_BACKTRACE=full for a verbose backtrace.
stack backtrace:
0: std::sys::windows::backtrace::unwind_backtrace
at C:\Dev\Projects\rust\src\libstd\sys\windows\backtrace\mod.rs:65
1: std::sys_common::backtrace::_print
at C:\Dev\Projects\rust\src\libstd\sys_common\backtrace.rs:71
2: std::sys_common::backtrace::print
at C:\Dev\Projects\rust\src\libstd\sys_common\backtrace.rs:58
3: std::panicking::default_hook::{{closure}}
at C:\Dev\Projects\rust\src\libstd\panicking.rs:380
4: std::panicking::default_hook
at C:\Dev\Projects\rust\src\libstd\panicking.rs:396
5: std::panicking::rust_panic_with_hook
at C:\Dev\Projects\rust\src\libstd\panicking.rs:576
6: std::panicking::begin_panic<str*>
at C:\Dev\Projects\rust\src\libstd\panicking.rs:537
7: proc_macro::__internal::with_sess
at C:\Dev\Projects\rust\src\libproc_macro\lib.rs:851
8: proc_macro::TokenTree::from_internal
at C:\Dev\Projects\rust\src\libproc_macro\lib.rs:682
9: proc_macro::{{impl}}::next
at C:\Dev\Projects\rust\src\libproc_macro\lib.rs:570
10: proc_macro2::imp::{{impl}}::next
at C:\Users\Rantanen\.cargo\registry\src\github.com-1ecc6299db9ec823\proc-macro2-0.2.2\src\unstable.rs:117
11: proc_macro2::{{impl}}::next
at C:\Users\Rantanen\.cargo\registry\src\github.com-1ecc6299db9ec823\proc-macro2-0.2.2\src\lib.rs:326
12: syn::buffer::TokenBuffer::inner_new
at C:\Users\Rantanen\.cargo\registry\src\github.com-1ecc6299db9ec823\syn-0.12.12\src\buffer.rs:176
13: syn::buffer::TokenBuffer::new2
at C:\Users\Rantanen\.cargo\registry\src\github.com-1ecc6299db9ec823\syn-0.12.12\src\buffer.rs:228
14: syn::synom::{{impl}}::parse2<fn(syn::buffer::Cursor) -> core::result::Result<(crate_level_proc_macro::Crate, syn::buffer::Cursor), syn::error::ParseError>,crate_level_proc_macro::Crate>
at C:\Users\Rantanen\.cargo\registry\src\github.com-1ecc6299db9ec823\syn-0.12.12\src\synom.rs:221
15: syn::parse2<crate_level_proc_macro::Crate>
at C:\Users\Rantanen\.cargo\registry\src\github.com-1ecc6299db9ec823\syn-0.12.12\src\lib.rs:601
16: syn::parse<crate_level_proc_macro::Crate>
at C:\Users\Rantanen\.cargo\registry\src\github.com-1ecc6299db9ec823\syn-0.12.12\src\lib.rs:580
17: crate_level_proc_macro::attribute
at C:\Dev\Projects\crate_level_proc_macro\src\lib.rs:54
18: std::panicking::try::do_call
19: _rust_maybe_catch_panic
21: <syntax_ext::proc_macro_impl::AttrProcMacro as syntax::ext::base::AttrProcMacro>::expand
22: syntax::ext::expand::MacroExpander::expand
23: syntax::ext::expand::MacroExpander::expand
24: syntax::ext::expand::MacroExpander::expand_crate
25: rustc_driver::driver::count_nodes
26: rustc_driver::driver::count_nodes
27: rustc_driver::driver::compile_input
28: rustc_driver::run_compiler
error: custom attribute panicked
--> src\main.rs:2:1
|
2 | #![::crate_level_proc_macro::attribute]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= help: message: proc_macro::__internal::with_sess() called before set_parse_sess()!


The fact that the same code breaks on my own stage-1 rustc build based on the commit 29c8276ce, while it "works" on nightly 29c8276ce 2018-02-07 is a bit weird. Shouldn't those two compilers have the same code base?

In any case, I can try working around that specific issue by not going through proc_macro2 crate and instead using Syn's parse_str. However...

## .to_string() on TokenStream ❌

We can skip proc_macro2 by using the TokenStream::to_string() to turn the stream into rust-code that Syn can parse on its own. This gives us the following attribute function:

#[proc_macro_attribute]
pub fn attribute(_attr: TokenStream, input: TokenStream) -> TokenStream {
let file : Crate = parse_str( &input.to_string() ).unwrap();
quote!( #file ).into()
}

Running this results in the following panic

thread '<unnamed>' panicked at 'index out of bounds: the len is 0 but the index is 0',

   Compiling test_crate v0.1.0 (file:///C:/Dev/Projects/crate_level_proc_macro/test_crate)
thread '<unnamed>' panicked at 'index out of bounds: the len is 0 but the index is 0', C:\Dev\Projects\rust\src\liballoc\vec.rs:1551:10
note: Some details are omitted, run with RUST_BACKTRACE=full for a verbose backtrace.
stack backtrace:
0: std::sys::windows::backtrace::unwind_backtrace
at C:\Dev\Projects\rust\src\libstd\sys\windows\backtrace\mod.rs:65
1: std::sys_common::backtrace::_print
at C:\Dev\Projects\rust\src\libstd\sys_common\backtrace.rs:71
2: std::sys_common::backtrace::print
at C:\Dev\Projects\rust\src\libstd\sys_common\backtrace.rs:58
3: std::panicking::default_hook::{{closure}}
at C:\Dev\Projects\rust\src\libstd\panicking.rs:380
4: std::panicking::default_hook
at C:\Dev\Projects\rust\src\libstd\panicking.rs:396
5: std::panicking::rust_panic_with_hook
at C:\Dev\Projects\rust\src\libstd\panicking.rs:576
6: std::panicking::begin_panic<alloc::string::String>
at C:\Dev\Projects\rust\src\libstd\panicking.rs:537
7: std::panicking::begin_panic_fmt
at C:\Dev\Projects\rust\src\libstd\panicking.rs:521
8: std::panicking::rust_begin_panic
at C:\Dev\Projects\rust\src\libstd\panicking.rs:497
9: core::panicking::panic_fmt
at C:\Dev\Projects\rust\src\libcore\panicking.rs:71
10: core::panicking::panic_bounds_check
at C:\Dev\Projects\rust\src\libcore\panicking.rs:58
11: alloc::vec::{{impl}}::index
at C:\Dev\Projects\rust\src\liballoc\vec.rs:1551
12: syntax_pos::span_encoding::SpanInterner::get
at C:\Dev\Projects\rust\src\libsyntax_pos\span_encoding.rs:134
13: syntax_pos::span_encoding::decode::{{closure}}
at C:\Dev\Projects\rust\src\libsyntax_pos\span_encoding.rs:109
14: syntax_pos::span_encoding::with_span_interner::{{closure}}
at C:\Dev\Projects\rust\src\libsyntax_pos\span_encoding.rs:144
17: syntax_pos::span_encoding::with_span_interner
at C:\Dev\Projects\rust\src\libsyntax_pos\span_encoding.rs:144
18: syntax_pos::span_encoding::decode
at C:\Dev\Projects\rust\src\libsyntax_pos\span_encoding.rs:109
19: syntax_pos::span_encoding::Span::data
at C:\Dev\Projects\rust\src\libsyntax_pos\span_encoding.rs:47
20: syntax_pos::span_encoding::Span::lo
at C:\Dev\Projects\rust\src\libsyntax_pos\lib.rs:196
21: syntax::print::pprust::State::print_item
at C:\Dev\Projects\rust\src\libsyntax\print\pprust.rs:1173
22: syntax::print::pprust::item_to_string::{{closure}}
at C:\Dev\Projects\rust\src\libsyntax\print\pprust.rs:332
23: syntax::print::pprust::to_string
at C:\Dev\Projects\rust\src\libsyntax\print\pprust.rs:165
24: syntax::print::pprust::item_to_string
at C:\Dev\Projects\rust\src\libsyntax\print\pprust.rs:331
25: syntax::print::pprust::token_to_string
at C:\Dev\Projects\rust\src\libsyntax\print\pprust.rs:269
26: syntax::print::pprust::PrintState::print_tt<syntax::print::pprust::State>
at C:\Dev\Projects\rust\src\libsyntax\print\pprust.rs:794
27: syntax::print::pprust::PrintState::print_tts<syntax::print::pprust::State>
at C:\Dev\Projects\rust\src\libsyntax\print\pprust.rs:818
28: syntax::print::pprust::tokens_to_string::{{closure}}
at C:\Dev\Projects\rust\src\libsyntax\print\pprust.rs:320
29: syntax::print::pprust::to_string
at C:\Dev\Projects\rust\src\libsyntax\print\pprust.rs:165
30: syntax::print::pprust::tokens_to_string
at C:\Dev\Projects\rust\src\libsyntax\print\pprust.rs:320
31: syntax::tokenstream::{{impl}}::fmt
at C:\Dev\Projects\rust\src\libsyntax\tokenstream.rs:555
32: core::fmt::{{impl}}::fmt<proc_macro::TokenStream>
at C:\Dev\Projects\rust\src\libcore\fmt\mod.rs:1566
33: core::fmt::Formatter::run
at C:\Dev\Projects\rust\src\libcore\fmt\mod.rs:1084
34: core::fmt::write
at C:\Dev\Projects\rust\src\libcore\fmt\mod.rs:1030
35: core::fmt::Write::write_fmt<alloc::string::String>
at C:\Dev\Projects\rust\src\libcore\fmt\mod.rs:226
36: alloc::string::{{impl}}::to_string<proc_macro::TokenStream>
at C:\Dev\Projects\rust\src\liballoc\string.rs:2054
37: crate_level_proc_macro::attribute
at C:\Dev\Projects\crate_level_proc_macro\src\lib.rs:54
38: std::panicking::try::do_call
39: _rust_maybe_catch_panic
41: <syntax_ext::proc_macro_impl::AttrProcMacro as syntax::ext::base::AttrProcMacro>::expand
42: syntax::ext::expand::MacroExpander::expand
43: syntax::ext::expand::MacroExpander::expand
44: syntax::ext::expand::MacroExpander::expand_crate
45: rustc_driver::driver::count_nodes
46: rustc_driver::driver::count_nodes
47: rustc_driver::driver::compile_input
48: rustc_driver::run_compiler
error: custom attribute panicked
--> src\main.rs:2:1
|
2 | #![::crate_level_proc_macro::attribute]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: Could not compile test_crate.


I suspect this has something to do with the token stream including those pub mod bits, which do not have proper span information. The panic seems to result from the pretty printer trying to fetch SpanData for tokens that don't have any.

That's as far as I've gotten with this. I still believe crate-level macro attributes are a strong feature that is worth pursuing.

My own use case for these is to emit some FFI entry point functions, which enable the crate to be used as a COM library. While these could be done from a normal macro, an attribute would make it feel more declarative.

These attributes also came up in custom test frameworks. A while back I was playing around with a Catch-inspired ratcc test framework, where I would have loved to to use a crate-level attribute to alter the crate contents.

And continuing on the test framework theme, they also came up in the eRFC discussion for custom test frameworks: rust-lang/rfcs#2318

Given the amount of different panics I encountered in trying to work around the issues hints that they are currently far from supported. However the fact that an identity attribute works makes me hopeful that if the parsing could be fixed so that a TokenStream could be parsed into an ast::Crate (or similar), then the rest of the macro pipeline would "just work".

mentioned this issue Feb 8, 2018
changed the title Attribute macros invoked at crate root fail to resolve Attribute macros invoked at crate root have issues Feb 8, 2018

### abonander commented Apr 4, 2018

 A crate's token-stream should look exactly like the source; we shouldn't be emitting pub mod {} around its contents.

### abonander commented Apr 4, 2018 • edited

 I think the solution here would be to introduce Crate variants for Expansion[Kind]/Annotatable/Nonterminal and handle them accordingly. However, syn would have to know to parse multiple items or else most users won't see the whole input.

### abonander commented Apr 4, 2018

 Or would it be preferable to pass a syntatically correct representation of the crate as some pub mod krate {}?

### Rantanen commented Apr 4, 2018

 However, syn would have to know to parse multiple items or else most users won't see the whole input. I believe syn already handles "crate" properly as https://docs.rs/syn/0.12/syn/struct.File.html Or would it be preferable to pass a syntatically correct representation of the crate as some pub mod krate {}? From a user perspective, I found the pub mod {} a bit confusing at first, given those tokens do not exist in the parsed input. However if this issue didn't exist and I could have just used syn, I would have never even encountered the pub mod { .. } syntax as a user. So while a token stream without pub mod {} would be more pure, I don't know if it would be worth a much more complex implementation at least from user perspective.

### abonander commented Apr 4, 2018

 I think for sanity's sake we should pass the crate's tokens just as they appear in the source, which would mean handling a crate not like a module but like its own thing.

This was referenced Apr 18, 2018

### abonander commented Apr 27, 2018

 In #50101 @alexcrichton stated that he doesn't expect support for proc-macros on the crate root to be stabilized in the foreseeable future, that we should address the possible use-cases individually instead of allowing an attribute to completely rewrite the crate.

### Rantanen commented Apr 27, 2018

 The use case that prompted me to examine this in the first place was prototyping parts of the test framework pre-RFC. That RFC suggested a whole-crate proc-macro as one option for custom test framework. It even goes as far as stating the following for one of the options: This assumes that #![foo] ("inner attribute") macros work on modules and on crates. Currently the general understanding among people seems to be that inner attribute proc macros not working on modules and crates is a bug instead of an unsupported feature. If I remember right they do work on modules after all; Allowing people to wrap the whole crate in a mod statement to achieve the same result. Though I do appreciate @alexcrichton's concerns over this issue. Just to keep the discussion here I'll copy his response from #50101 here: Er sorry yeah, let me clarify. Right now for "Macros 1.2" were very unlikely to stabilize attribute invocations on modules. It's a weird question of what does #[foo] mod foo; receive? Does it receive mod foo ;? The contents of the module foo? Something like entire crate expansion also has huge repercussions on hygiene as well as whether it's feasible/quick. Entire crate expansion runs a risk of being extremely slow (we have to serialize back and forth with the procedural macro) and highly non-incremental (it's a complete black box to the compiler). I'd imagine that we're very far away from stabilizing this functionality, if at all. In that sense I'd personally be wary of landing this functionality in the compiler, even on nightly. I think it'd be best to take other avenues of attack for use cases that would otherwise require entire-crate expansion.

### Manishearth commented Apr 27, 2018

 FWIW for the test frameworks thing we can make it work without explicitly having this feature, what test frameworks need is a way to apply a whole-crate proc macro _as an argument to rustc_. This is less tricky. … On Fri, Apr 27, 2018, 2:01 AM Mikko Rantanen ***@***.***> wrote: The use case that prompted me to examine this in the first place was prototyping parts of the test framework pre-RFC. That RFC suggested a whole-crate proc-macro as one option for custom test framework. It even goes as far as stating the following for one of the options: This assumes that #![foo] ("inner attribute") macros work on modules and on crates. Currently the general understanding among people seems to be that inner attribute proc macros not working on modules and crates is a bug instead of an unsupported feature. If I remember right they do work on modules after all; Allowing people to wrap the whole crate in a mod statement to achieve the same result. Though I do appreciate @alexcrichton 's concerns over this issue. Just to keep the discussion here I'll copy his response from #50101 <#50101> here: Er sorry yeah, let me clarify. Right now for "Macros 1.2" were very unlikely to stabilize attribute invocations on modules. It's a weird question of what does #[foo] mod foo; receive? Does it receive mod foo ;? The contents of the module foo? Something like entire crate expansion also has huge repercussions on hygiene as well as whether it's feasible/quick. Entire crate expansion runs a risk of being extremely slow (we have to serialize back and forth with the procedural macro) and highly non-incremental (it's a complete black box to the compiler). I'd imagine that we're very far away from stabilizing this functionality, if at all. In that sense I'd personally be wary of landing this functionality in the compiler, even on nightly. I think it'd be best to take other avenues of attack for use cases that would otherwise require entire-crate expansion. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#41430 (comment)>, or mute the thread .

### Rantanen commented Apr 27, 2018

 @Manishearth Doesn't that result in the same hygiene/incremental issues? Referring here to @alexcrichton's comment: Something like entire crate expansion also has huge repercussions on hygiene as well as whether it's feasible/quick. Entire crate expansion runs a risk of being extremely slow (we have to serialize back and forth with the procedural macro) and highly non-incremental (it's a complete black box to the compiler). I know one of the options for the test framework RFC is to just generate a main function. The "Only add stuff"-kind of proc macro does somewhat circumvent the hygiene/incrmental compilation issues, but that's just one option mentioned in the RFC - all the other options face the same issues I'd imagine. Essentially what I'm after here is: If those issues are solved/resolved/accepeted in relation to the the test RFC, doesn't that apply here as well? Although what I suspect is that given the issue raised here, the test RFC will end up going with the "just generate main()" option.

### Manishearth commented Apr 27, 2018

 Slow and non-incremental matters less for testing. "just add a main function" is none of the options, all of the options already have to run some kind of whole crate proc macro to rewrite things to be pub. This is what libtest already does, it folds the entire crate. The question debated in the RFC is whether or not this should be exposed to the user as a whole crate proc macro API. The hygiene issue is still a thing, but if folks just use the "generate main" API we put on crates.io that's still not a problem. And most of the frameworks which wish to do something more complex than that probably won't need to worry as much about hygiene.

### Manishearth commented Apr 27, 2018

 Well, that's not entirely true -- the "just generate main" API would have less of a serialization overhead. But that's about it.

### alexcrichton commented Apr 27, 2018

 Another reason that I don't think that this is a great idea is #43081 right now. Any change to the AST would lose span information for the entire crate. I think that's basically a showstopper until we fix that.

mentioned this issue Oct 22, 2018
mentioned this issue Jan 27, 2019
mentioned this issue Aug 27, 2019
added A-attributes T-compiler labels Oct 4, 2019
mentioned this issue Oct 8, 2019

### vakaras commented Feb 29, 2020

 This is what libtest already does, it folds the entire crate. Could someone point me to the code that does this? (I could not find it myself.) We want to update the Prusti verifier to work with the latest version of the Rust compiler. However, one of the blocking issues we have is that I cannot find any more a way to obtain a mutable reference into an AST. We used to rewrite AST to type-check specifications. Treating each specification as a separate procedural macro invocation is very problematic because we need to maintain a global state.

### Robbepop commented Nov 18, 2020

 Since #43081 (comment) mentions that #43081 is likely to be fixed soon for all stable Rust syntax we can reiterate on this issue. I'd love to hear what else is blocking this feature and if there are any other show stoppers, bugs or design considerations before we can get there.