resolve: Implement prelude search for macro paths, implement tool attributes

[petrochenkov]

When identifier is macro path is resolved in scopes (i.e. the first path segment - foo in foo::mac!() or foo!()), scopes are searched in the same order as for non-macro paths - items in modules, extern prelude, tool prelude (see later), standard library prelude, language prelude, but with some extra shadowing restrictions (names from globs and macro expansions cannot shadow names from outer scopes). See the comment in fn resolve_lexical_macro_path_segment for more details.

"Tool prelude" currently contains two "tool modules" rustfmt and clippy, and is searched immediately after extern prelude.
This makes the possible long-term solution for tool attributes exactly equivalent to the existing extern prelude scheme, except that --extern=my_crate making crate names available in scope is replaced with something like --tool=my_tool making tool names available in scope.

The tool_attributes feature is still unstable and #![feature(tool_attributes)] now implicitly enables #![feature(use_extern_macros)]. use_extern_macros is a prerequisite for tool_attributes, so their stabilization will happen in the same order.
If use_extern_macros is not enabled, then tool attributes are treated as custom attributes (this is temporary, anyway).

Fixes #52576
Fixes #52512
Fixes #51277
cc #52269

[rust-highfive]

r? @cramertj

(rust_highfive has picked a reviewer for you, use r? to override)

[petrochenkov]

r? @alexcrichton
cc @nrc @djrenren

[petrochenkov]

FIXME: account for no_implicit_prelude.
Done.

[rust-highfive]

The job x86_64-gnu-llvm-5.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

[01:18:25] 
[01:18:25] failures:
[01:18:25] 
[01:18:25] ---- /checkout/src/doc/unstable-book/src/language-features/tool-attributes.md - tool_attributes::An_example (line 17) stdout ----
[01:18:25] error[E0658]: The attribute `rustfmt::skip` is currently unknown to the compiler and may have meaning added to it in the future (see issue #29642)
[01:18:25]   |
[01:18:25] 4 | #[rustfmt::skip]
[01:18:25]   | ^^^^^^^^^^^^^^^^
[01:18:25]   |
[01:18:25]   |
[01:18:25]   = help: add #![feature(custom_attribute)] to the crate attributes to enable
[01:18:25] thread '/checkout/src/doc/unstable-book/src/language-features/tool-attributes.md - tool_attributes::An_example (line 17)' panicked at 'couldn't compile the test', librustdoc/test.rs:332:13
[01:18:25] note: Run with `RUST_BACKTRACE=1` for a backtrace.
[01:18:25] 
[01:18:25] 

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

[alexcrichton]

Looks great to me, thanks for the detailed comments!

It looks like tool attributes like #[rustfmt::skip] are still gated by default, and otherwise it looks like this is just fixing bugs, so r=me!

[petrochenkov]

@bors r=alexcrichton

[bors]

📌 Commit dd93535 has been approved by alexcrichton

[Mark-Simulacrum]

@bors p=1 edition critical

[bors]

🔒 Merge conflict

This pull request and the master branch diverged in a way that cannot be automatically merged. Please rebase on top of the latest master branch, and let the reviewer approve again.

How do I rebase?

Assuming self is your fork and upstream is this repository, you can resolve the conflict following these steps:

1. git checkout premacro (switch to your branch)
2. git fetch upstream master (retrieve the latest master)
3. git rebase upstream/master -p (rebase on top of it)
4. Follow the on-screen instruction to resolve conflicts (check git status if you got lost).
5. git push self premacro --force-with-lease (update this PR)

You may also read Git Rebasing to Resolve Conflicts by Drew Blessing for a short tutorial.

Please avoid the "Resolve conflicts" button on GitHub. It uses git merge instead of git rebase which makes the PR commit history more difficult to read.

Sometimes step 4 will complete without asking for resolution. This is usually due to difference between how Cargo.lock conflict is handled during merge and rebase. This is normal, and you should still perform step 5 to update this PR.

Error message

Auto-merging src/test/ui/macro-path-prelude-pass.rs
Auto-merging src/test/ui/macro-path-prelude-fail-1.rs
Auto-merging src/librustc_resolve/macros.rs
CONFLICT (content): Merge conflict in src/librustc_resolve/macros.rs
Auto-merging src/librustc_resolve/lib.rs
Auto-merging src/librustc_resolve/build_reduced_graph.rs
Automatic merge failed; fix conflicts and then commit the result.

[pietroalbini]

This is also blocking edition crater runs.

[petrochenkov]

@bors r=alexcrichton

[bors]

📌 Commit c3e5421 has been approved by alexcrichton

[bors]

⌛️ Testing commit c3e5421 with merge e53bca8...

[pietroalbini]

@bors r=alexcrichton

[bors]

📌 Commit c3e5421 has been approved by alexcrichton

[pietroalbini]

@bors p=1

[alexcrichton]

@bors: rollup-

[bors]

⌛️ Testing commit c3e5421 with merge 40e4b6e...

[bors]

☀️ Test successful - status-appveyor, status-travis
Approved by: alexcrichton
Pushing 40e4b6e to master...

[Manishearth]

This breaks Servo in a weird way. I'm unable to reproduce this out of tree

https://travis-ci.org/servo/servo-with-rust-nightly/jobs/411851355

error[E0659]: `warn` is ambiguous
   --> ports/servo/glutin_app/window.rs:733:21
    |
733 |                     warn!("Failed to swap window buffers ({}).", err);
    |                     ^^^^
    |
note: `warn` could refer to the name imported here
   --> ports/servo/non_android_main.rs:12:1
    |
12  | #[macro_use] extern crate log;
    | ^^^^^^^^^^^^
note: `warn` could also refer to the name defined here
   --> ports/servo/glutin_app/window.rs:733:21
    |
733 |                     warn!("Failed to swap window buffers ({}).", err);
    |                     ^^^^
    = note: macro-expanded macro imports do not shadow

any idea what's going on? seems like a regression

[Manishearth]

cc @pietroalbini might want to be careful about that regression before cutting a beta ^^

[petrochenkov]

I'll look what happens.

[petrochenkov]

@Manishearth
This is an expected breakage.
The general rule sounds like "names from globs and macro expansions cannot shadow names from outer scopes when macro paths are resolved".
Without this rule macro expansion would almost always get stuck in scopes with unexpanded macros or globs without being able to proceed.

In this case macro expanded name (macro_rules macro warn from #[macro_use] extern crate log; inside include!() expansion) shadows a name from outer scope (built-in attribute warn from the language prelude), and this happens during resolution of the macro path warn in warn!("Failed to swap window buffers ({}).", err);.

Minimized reproduction:

macro_rules! my_include {() => {
    #[macro_use] extern crate log;
}}

my_include!();

fn main() {
    warn!("");
}

In this case, since breakage affects stable channel, built-in attributes can be somehow special-cased to avoid the error, or perhaps some more general solution can be found, but I'd prefer to see crater results before proceeding with a fix.

[petrochenkov]

Since #53072 doesn't include this PR, we need a separate crater run for it.

[Mark-Simulacrum]

Queued a crater run for this PR (check only) here: #53089

[nnethercote]

Good news! This PR improved compile speed on a few benchmarks, the best by 3.7%:
https://perf.rust-lang.org/compare.html?start=40cb4478a3f550bf12d81cdcf08cc9ef3985ed41&end=40e4b6ee3dd70a05007915bd1c15c150a7b7899f&stat=instructions:u