Stabilize `use_extern_macros`

[petrochenkov]

Closes #35896

[rust-highfive]

r? @cramertj

(rust_highfive has picked a reviewer for you, use r? to override)

[petrochenkov]

r? @kennytm

[petrochenkov]

@bors try

[bors]

⌛️ Trying commit 66dfde1 with merge ea32c49...

[bors]

☀️ Test successful - status-travis
State: approved= try=True

[kennytm]

cc @rust-lang/infra Crater run (check only) requested.

[pietroalbini]

Crater run (check only) started.

[pietroalbini]

Hi @kennytm (crater requester, PR reviewer)! Crater results are at: http://cargobomb-reports.s3.amazonaws.com/pr-50911/index.html. 'Blacklisted' crates (spurious failures etc) can be found here. If you see any spurious failures not on the list, please make a PR against that file.

(interested observers: Crater is a tool for testing the impact of changes on the crates.io ecosystem. You can find out more at the repo if you're curious)

[kennytm]

6 regressions.

• verilog: Seems to be spurious ("Read-only file system" cc @pietroalbini).
• npbot: This PR seems to have broken its use of structopt-^0.2.5 via quicli-0.2.0. Legit.
• The rest: All due to #[macro_use] no longer needed, which causes #[deny(warnings)] to error out. Legit but expected.

Relevant source code in npbot-1.0.0

[dependencies]
structopt = "0.2.5"

#[macro_use]
extern crate quicli;

// ...

use structopt::StructOpt;
use quicli::prelude::*;

// ...

#[derive(StructOpt)]
pub(crate) struct Opt {
    #[structopt(short = "d", long = "debug", help = "only use the local, println!() sink")]
    debug: bool,
    #[structopt(long = "verbose", short = "v", parse(from_occurrences))]
    verbose: u8,

    // for the SoundTouch source
    #[structopt(long = "soundtouch-host")]
    soundtouch_host: Option<String>,

    // for the last.fm source
    #[structopt(long = "lastfm-api-key")]
    lastfm_api_key: Option<String>,
    #[structopt(long = "lastfm-username")]
    lastfm_username: Option<String>,
}

Note the lack of #[macro_use] extern crate structopt.

Error log

May 22 22:49:57.928 INFO kablam! error[E0658]: The attribute `structopt` is currently unknown to the compiler and may have meaning added to it in the future (see issue #29642)
May 22 22:49:57.928 INFO kablam!   --> src/main.rs:41:5
May 22 22:49:57.928 INFO kablam!    |
May 22 22:49:57.928 INFO kablam! 41 |     #[structopt(short = "d", long = "debug", help = "only use the local, println!() sink")]
May 22 22:49:57.928 INFO kablam!    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 22 22:49:57.928 INFO kablam!    |
May 22 22:49:57.929 INFO kablam!    = help: add #![feature(custom_attribute)] to the crate attributes to enable
May 22 22:49:57.929 INFO kablam! 
May 22 22:49:57.929 INFO kablam! error[E0658]: The attribute `structopt` is currently unknown to the compiler and may have meaning added to it in the future (see issue #29642)
May 22 22:49:57.929 INFO kablam!   --> src/main.rs:43:5
May 22 22:49:57.929 INFO kablam!    |
May 22 22:49:57.929 INFO kablam! 43 |     #[structopt(long = "verbose", short = "v", parse(from_occurrences))]
May 22 22:49:57.929 INFO kablam!    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 22 22:49:57.929 INFO kablam!    |
May 22 22:49:57.929 INFO kablam!    = help: add #![feature(custom_attribute)] to the crate attributes to enable
May 22 22:49:57.929 INFO kablam! 
May 22 22:49:57.929 INFO kablam! error[E0658]: The attribute `structopt` is currently unknown to the compiler and may have meaning added to it in the future (see issue #29642)
May 22 22:49:57.929 INFO kablam!   --> src/main.rs:47:5
May 22 22:49:57.929 INFO kablam!    |
May 22 22:49:57.929 INFO kablam! 47 |     #[structopt(long = "soundtouch-host")]
May 22 22:49:57.929 INFO kablam!    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 22 22:49:57.929 INFO kablam!    |
May 22 22:49:57.929 INFO kablam!    = help: add #![feature(custom_attribute)] to the crate attributes to enable
May 22 22:49:57.929 INFO kablam! 
May 22 22:49:57.929 INFO kablam! error[E0658]: The attribute `structopt` is currently unknown to the compiler and may have meaning added to it in the future (see issue #29642)
May 22 22:49:57.929 INFO kablam!   --> src/main.rs:51:5
May 22 22:49:57.929 INFO kablam!    |
May 22 22:49:57.929 INFO kablam! 51 |     #[structopt(long = "lastfm-api-key")]
May 22 22:49:57.929 INFO kablam!    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 22 22:49:57.929 INFO kablam!    |
May 22 22:49:57.929 INFO kablam!    = help: add #![feature(custom_attribute)] to the crate attributes to enable
May 22 22:49:57.929 INFO kablam! 
May 22 22:49:57.930 INFO kablam! error[E0658]: The attribute `structopt` is currently unknown to the compiler and may have meaning added to it in the future (see issue #29642)
May 22 22:49:57.930 INFO kablam!   --> src/main.rs:53:5
May 22 22:49:57.930 INFO kablam!    |
May 22 22:49:57.930 INFO kablam! 53 |     #[structopt(long = "lastfm-username")]
May 22 22:49:57.930 INFO kablam!    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 22 22:49:57.930 INFO kablam!    |
May 22 22:49:57.930 INFO kablam!    = help: add #![feature(custom_attribute)] to the crate attributes to enable
May 22 22:49:57.930 INFO kablam! 
May 22 22:49:57.931 INFO kablam! error: aborting due to 5 previous errors

[petrochenkov]

I'll look what happens with npbot.

[petrochenkov]

FIXME: possible_time_travel is now unused

[petrochenkov]

It looks like npbot legitimately encounters a case of indeterminate resolution when macro expansion cannot progress and generates dummy fragments to recover.

So the error should be something like "cannot determine resolution for the derive macro StructOpt".

However, due to a scary hole in our macro resolution checking such dummy expansions can happen silently without generating any errors.
So we are seeing only secondary errors about unknown structopt attributes instead of the actual error about unresolved StructOpt.
I'll prepare a fix.

[petrochenkov]

Fixed in #51145

[petrochenkov]

r? @alexcrichton

[bors]

☀️ Test successful - status-appveyor, status-travis
Approved by: alexcrichton
Pushing c8c587f to master...

[alexcrichton]

🎊

Thanks so much for helping this over the finish line @petrochenkov!

[softprops]

The 1.30.0 release notes for #[macro_export(local_inner_macros)] link here.

Where can I find more documentation on this feature?

So far I've tried local_inner_macros in two cases where a crate's macros depended on anothers and it didn't work in either case.

ilianaw/rust-crowbar#46 (comment)

dtolnay/mashup#16

[petrochenkov]

@softprops
rust-lang-nursery/edition-guide#117

It will be available at https://rust-lang-nursery.github.io/edition-guide/rust-2018/macros/macro-changes.html when merged.

[softprops]

@petrochenkov thanks! I'll take a look and report back if it looks like what I'm trying to do, re-export transitive dependency crates macros` is actually what this new feature supports.

[softprops]

@petrochenkov l need to do a bit more testing but I worked it out, at least enough to get a fork of rust crowbar to compile. I still need to test a crate that uses that fork before I can call this 👍 'd

It may not be clear to new rust users that for foreign macros, you'll also need namespace the macro's path. In my case I just needed to prefix crowbars use of cpython's macros with cpython:: in the docs you linked to and users will find you only use local macros in examples using $crate::. Assuming macros authors are already going to be a bit more advanced than the beginner rust users they may be able to discover this for themselves. I'd recommend adding at least one foreign macro example for the sake of beginner/advanced user inclusivity. Albeit this would be much more painful for the example of supporting old rust versions because if forces users to copy and paste the entire macro srcs from other crates. The format_args is probably the simplest case for a macro. A more complex examples will be much more awkward. For now, asking users to use the latest stable works well enough!

[softprops]

found a potential bug but I'm not sure if its by design

when experimenting with the $crate:: path prefix with #[macro_export(local_inner_macros)] and with #[macro_export] I found this interesting inconsistency gotcha with the way that exported macro can be used.

with #[macro_export(local_inner_macros)] I can do the following

extern crate crowbar;
crowbar::lambda!(...)

however with #[macro_export] I get the following here using the same path references

extern crate crowbar;
crowbar::lambda!(...)

error: cannot find macro `lambda!` in this scope

interestingly though I can do the following with #[macro_export]

extern crate crowbar;
use crowbar::lambda;
lambda!(...)

this feels a bit inconsistent in the way the newer system attempts to line up with what you can do with path references for non macro items.

[petrochenkov]

@softprops
I suspect that the lambda macro is recursive and calls itself by relative name (lambda!(...)), in other words, it's its own helper.

This way the first iteration crowbar::lambda!(...) is resolved successfully, but the next iterations cannot be resolved without bringing lambda in scope at use-site or using local_inner_macros at def-site.

[softprops]

@petrochenkov 100%. that was it. thanks!