[beta] resolve: Implement edition hygiene for imports and absolute paths

[petrochenkov]

The changes in behavior of imports and absolute paths are the most significant breaking changes of 2018 edition.
However, these changes are not covered by edition hygiene, so macros defined by 2015 edition crates expanded in 2018 edition crates are still interpreted in the 2018 edition way when they contain imports or absolute paths.
This means the promise of seamless integration of crates built with different editions, including use of macros, doesn't hold fully.
This PR fixes that and implements edition hygiene for imports and absolute paths, so they behave according to the edition in which they were written, even in macros.

Detailed rules (mostly taken from #50376)

Preface

We keep edition data per-span in the compiler. This means each span knows its edition.
Each identifier has a span, so each identifier knows its edition.

Absolute paths

Explicitly written absolute paths ::ident::... are desugared into something like {{root}}::ident::... in the compiler, where {{root}} is also treated as an identifier.
{{root}} inherits its span/hygienic-context from the token ::.

If the span is from 2015 edition, then {{root}} is interpreted as the current crate root (crate::... with same hygienic context).
If the span is from 2018 edition, then {{root}} is interpreted as "crate universe" (extern::...).

Imports

To resolve an import use ident::... we need to resolve ident first.
The idea in this PR is that ident fully knows how to resolve itself.

If ident's span is from 2015 edition, then the identifier is resolved in the current crate root (effectively crate::ident::... where crate has the same hygienic context as ident).
If ident's span is from 2018 edition, then the identifier is resolved in the current scope, without prepending anything (well, at least with uniform paths).

There's one corner case in which there's no identifier - prefix-less glob import use *;.
In this case the span is inherited from the token *.
use *; && is_2015(span(*)) -> use crate::*; && span(crate) == span(*).
use *; && is_2018(span(*)) -> error.

UPDATE

After crater run the rules were updated to include fallback from 2015 edition behavior to 2018 edition behavior, see #56053 (comment) for more details.

---

Why beta:
- Compatibility of 2015 edition crates with 2018 edition crates, including macros, is one of the main promises of the edition initiative.
- This is technically a breaking change for 2018 edition crates or crates depending on 2018 edition crates.
- This PR is based on #55884 which hasn't landed on nightly yet :) No longer true (#56042).

Previous attempt #50999
Closes #50376
Closes #52848
Closes #53007

r? @ghost

[petrochenkov]

r? @nikomatsakis
cc @rust-lang/lang

[petrochenkov]

It would be good to run crater on this if we still have time.
I wonder how many macro crates already rely on imports and absolute paths being edition-unhygienic.
cc @dtolnay @alexcrichton

@bors try

[bors]

⌛️ Trying commit 59d1c85 with merge 4f24eba...

[eddyb]

@petrochenkov How many other parts of the compiler check the "global" edition?
Would it be feasible to move all of them over?

[bors]

☀️ Test successful - status-travis
State: approved= try=True

[Centril]

The changes make a lot of sense. To get the ball rolling, pending crater run and code review (I mostly checked tests), I propose that we stabilize the behavior described in #56053 (comment).

[Centril]

@rfcbot merge

[rfcbot]

Team member @Centril has proposed to merge this. The next step is review by the rest of the tagged teams:

• @Centril
• @aturon
• @cramertj
• @eddyb
• @joshtriplett
• @nikomatsakis
• @nrc
• @pnkfelix
• @scottmcm
• @withoutboats

No concerns currently listed.

Once a majority of reviewers approve (and none object), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[joshtriplett]

Assuming crater passes: wow. Incredible work, @petrochenkov!

[petrochenkov]

@eddyb

How many other parts of the compiler check the "global" edition?

I'm not sure, will check today.
Edition-specific keywords are certainly covered, everything lint-related is probably non-critical, and NLL must be enabled globally.
Other changes between editions should be minor, IIRC.

[petrochenkov]

Assuming crater passes

What I'm concerned most is macro crates using imports and absolute paths to refer solely to extern crates and assuming extern crate items in the calling crate's root.

// Macro crate 2015 (procedural or declarative)
macro mac2015() {
    use my_crate::foo;
    fn do_things() {
        ::my_crate::bar();
    }
}

---

// User crate 2015

extern crate my_crate;

mac2015!();

---

// User crate 2018 (without hygiene)

mac2015!();

In this case despite the change in resolution behavior the macro still works on 2018 edition, and even in more idiomatic way - extern crate item in the root is not required.

• ::my_crate still refers to an extern crate with 2018 rules (but now in extern prelude rather than in the root)
• use my_crate refers to an extern crate with 2018 rules (but now in extern prelude rather than in the root), but now it's unreliable - it may be shadowed.

With edition hygiene as implemented in this PR, 2018 crates will still need an item referring to the extern crate in the root module when using 2015 macro crates using absolute paths and imports in this way.

// User crate 2018 (with hygiene)

use my_crate;

mac2015!();

(However, imports use my_crate::foo; become reliable again.)

[petrochenkov]

@craterbot run start=master#d08a0a89c6ed1d7c66cfbee2f85721a2d1696885 end=try#4f24eba9a9b426a262d22d2ce72a4139c3860eb6 mode=check-only

[craterbot]

👌 Experiment pr-56053 created and queued.
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[craterbot]

🚧 Experiment pr-56053 is now running on agent aws-1.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[petrochenkov]

Argh, there's much worse issue - 2015 macros using use std::foo; or ::std::foo.
With hygiene this desugars into use crate::std::foo/crate::std::foo.
The problem is that std no longer exists in the crate root on 2018 edition, it was removed by @eddyb in cd47831.

Possible solutions:

• Reintroduce std into the crate root on 2018 edition, but discourage its use through a lint or something.
  I think I'd prefer this solution because it also minimizes the behavior differences between editions and number of "global" edition checks.
  I'll prepare this variant for crater run.
• Make {{root}} edition-unhygienic, whether it's added explicitly or not.
  This way use foo::bar still desugars into use {{root}}::foo::bar if span(foo).is_2015(), but the {{root}} is interpreted differently on 2015 and 2018.
• Keep both absolute paths and imports unhygienic and close this PR.

@craterbot abort

[craterbot]

🗑 Experiment pr-56053 deleted!

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[Centril]

@petrochenkov in solution 1, would we also introduce crate::core:: for consistency then?

[petrochenkov]

@Centril
Yes, all automatically injected crates - std, core (depending on no-std options), and also compiler_builtins (but it's unstable to use, so it doesn't matter much).

[Centril]

@petrochenkov Seems like the cleanest solution to me also; and making it work with #![no_std] is great.

[petrochenkov]

One non-spurious regression (accept-encoding-0.1.0) with absolute_paths_not_starting_with_crate lint firing on a macro-expanded path.
I'll look what happens, but this is non-critical.
I'm not even sure the lint still does the right thing in general now when we have uniform paths.

@bors r=nikomatsakis,alexcrichton p=1

[bors]

📌 Commit 581b683 has been approved by nikomatsakis,alexcrichton

[bors]

💡 This pull request was already approved, no need to approve it again.

• There's another pull request that is currently being tested, blocking this pull request: #56240

[bors]

📌 Commit 581b683 has been approved by nikomatsakis,alexcrichton

[bors]

⌛️ Testing commit 581b683 with merge d33bb37...

[pietroalbini]

@bors retry r-

[rust-highfive]

Your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

Resolving deltas: 100% (7850/7850), done.
travis_time:end:008ac2ee:start=1543253255439045556,finish=1543253261293093851,duration=5854048295
$ cd rust-lang/rust
$ git checkout -qf d33bb3779c94a313e9d07bccc386258cc2d19ebb
fatal: reference is not a tree: d33bb3779c94a313e9d07bccc386258cc2d19ebb
The command "git checkout -qf d33bb3779c94a313e9d07bccc386258cc2d19ebb" failed and exited with 128 during .
Your build has been stopped.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

[pietroalbini]

Rolled up in #56240.

[bors]

☔️ The latest upstream changes (presumably #56240) made this pull request unmergeable. Please resolve the merge conflicts.