Switch to MIR-based translation by default.

[eddyb]

This patch makes -Z orbit default to "on", which means that by default, functions will be translated from Rust to LLVM IR through the upcoming MIR backend, instead of the antiquated AST backend.

This switch is made possible by the recently merged #33622, #33905 and smaller fixes.

If you experience any issues, please file a report for each of them. You can switch to the old backend to work around problems by either setting RUSTFLAGS="-Zorbit=off" or by annotating specific functions with #[rustc_no_mir] (which requires #![feature(rustc_attrs)] at the crate-level).

I would like this PR to get into nightly soon so that we can get early feedback in this release cycle and focus on correctness fixes and performance improvements, with the potential for removing the old backend implementation before beta branches off.

cc @rust-lang/compiler

[rust-highfive]

r? @pnkfelix

(rust_highfive has picked a reviewer for you, use r? to override)

[nagisa]

@bors r+

I feel like approving this PR for nightly makes a lot of sense. We’ve been testing and improving MIR for a while now and making it default on nightly will get MIR to many more hands than we had before so far.

One concern I have that as the PR is now its pretty easy to forget to flip the true to false before the branch to beta if we figure out that we are not ready to enable this yet. On the other hand, reversing enablement is as easy of an one-line-change, so @eddyb convinced me that it will be fine.

I hope time this PR will need to go through the queue will be enough for all relevant parties to see this.

[bors]

📌 Commit ce179ea has been approved by nagisa

[arielb1]

Sure. We have to flip that switch someday, and it's not like we will discover any new bugs with the switch set to off.

On the other hand, I would prefer a crater run with LLVM assertions enabled. I would prefer that to a dump of confusing ICE reports.

There are only 3 PRs left on the queue, and our build times improved since the opt-mir builds are no longer holding things up - this PR may go today UTC, and I am not sure @nikomatsakis is online.

cc @rust-lang/compiler

[arielb1]

hold hold hold
@bors r-

canceling pending a crater run

[nagisa]

… and niko’s seal of approval, of course, which implies

r? @nikomatsakis

[alexbool]

Whoa, how about #33828?

[nagisa]

I personally do not see performance regressions as a blocker to enable MIR in nightly by default, because otherwise we will never end up enabling it at all. Rather, enabling MIR by default on nightly would serve to help collect more reports on issues with MIR (both performance and correctness) and speed up fixing said issues. Perhaps all within the same cycle.

[eddyb]

@alexbool Those numbers predate non-zeroing drops. I can take a look, I suppose, maybe it has a completely different reason.

[alexbool]

@eddyb How is that related to non-zeroing drops? If I get it right, MIR and old code both use the same zeroing drops in that benchmark, so the competion is fair. (Please tell me if I am wrong)

[eddyb]

@alexbool Old trans can actually handle some static drop/unconditional move cases better than MIR trans did before #33622 - MIR trans used to generate a ton of memsets with the "uninitialized" and "dropped" byte patterns instead.

[alexbool]

Thanks, sounds convincing

[eddyb]

The Travis failure is strange, although likely legitimate:

run doc-book-lang-items [x86_64-unknown-linux-gnu]

running 1 test
test _0 ... FAILED

failures:

---- _0 stdout ----
    <anon>:36:9: 36:13 warning: unused variable: `argc`, #[warn(unused_variables)] on by default
<anon>:36 fn main(argc: isize, argv: *const *const u8) -> isize {
                  ^~~~
<anon>:36:22: 36:26 warning: unused variable: `argv`, #[warn(unused_variables)] on by default
<anon>:36 fn main(argc: isize, argv: *const *const u8) -> isize {
                               ^~~~
<anon>:37:9: 37:10 warning: unused variable: `x`, #[warn(unused_variables)] on by default
<anon>:37     let x = box 1;
                  ^
error: linking with `cc` failed: exit code: 1
note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/tmp/rustdoctest.goiITNucvd13/rust_out.0.o" "-o" "/tmp/rustdoctest.goiITNucvd13/rust_out" "-Wl,--gc-sections" "-pie" "-nodefaultlibs" "-L" "/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "-Wl,-Bdynamic" "/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-fe3cdf61.rlib" "/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-fe3cdf61.rlib" "-l" "c" "-l" "m" "-l" "rt" "-l" "util" "-l" "compiler-rt"
note: /tmp/rustdoctest.goiITNucvd13/rust_out.0.o: In function `rust_out::main::hec68da2489a63b6a':
rust_out.0.rs:(.text._ZN8rust_out4main17hec68da2489a63b6aE+0x6a): undefined reference to `_Unwind_Resume'
collect2: error: ld returned 1 exit status

error: aborting due to previous error
thread '_0' panicked at 'Box<Any>', src/libsyntax/errors/mod.rs:622
note: Run with `RUST_BACKTRACE=1` for a backtrace.
thread '_0' panicked at 'couldn't compile the test', src/librustdoc/test.rs:281


failures:
    _0

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured

/build/mk/tests.mk:865: recipe for target 'tmp/check-stage2-T-x86_64-unknown-linux-gnu-H-x86_64-unknown-linux-gnu-doc-book-lang-items.ok' failed
make: *** [tmp/check-stage2-T-x86_64-unknown-linux-gnu-H-x86_64-unknown-linux-gnu-doc-book-lang-items.ok] Error 101

@alexcrichton Is it possible --enable-orbit doesn't work for everything we run, in this case rustdoc?

[eddyb]

@alexcrichton The example seems fundamentally broken and only happens to work with old trans because no landing pads get generated for that exact code. Although the fix appears to be simple so I'll just include it here.

[alexcrichton]

@eddyb oh we probably don't pass RUSTFLAGS to the rustdoc unit tests (or rustdoc also just doesn't accept -Z), explaining at least why we didn't see it before.

@eddyb also yeah I'm fine with mangling that test or otherwise just ignoring it, it seems like a difficult thing to test anyway. Having it as part of rustdoc also probably isn't helping us much...

[nikomatsakis]

Let's discuss a bit here perhaps: http://internals.rust-lang.org/t/enabling-mir-by-default/3555 ?

[eddyb]

@NastyRigger I have one data point for (compiler) memory usage: 625MB -> 604MB from old trans to MIR trans, on stage1 libsyntax, just after LLVM passes.

[brson]

I personally do not see performance regressions as a blocker to enable MIR in nightly by default, because otherwise we will never end up enabling it at al

Once MIR is on the likelyhood of us turning it back on seems very low, so I have to disagree with this. We must hold the line on performance - we have already regressed compile time performance badly this year. Do we have evidence that compile time, run time and code size are on par with oldtrans?

[MagaTailor]

@eddyb @brson I remember comparing binary sizes of parity and rust's lib directory a few weeks ago. I've just finished repeating the tests, and to my utter surprise, instead of 10% larger, MIR produced a 5% smaller parity binary, whereas the lib directory ended up 1.7% smaller. (32-bit ARM data)

During the latter test /usr/bin/time -v make -j2 showed maximum resident memory size of 543124 kB vs 391828 kB (probably librustc), in favour of MIR again. I wasn't able to benchmark compilation times but other than that, MIR seems to have improved immensely!

Edit: Cargo binary is made 8% smaller.

[eddyb]

@brson Actually, I mentioned libsyntax using less memory to compile, here's the timings/RSS:
Old trans:

time: 11.669; rss: 734MB        translation
  time: 4.534; rss: 621MB       llvm function passes [0]
  time: 94.234; rss: 623MB      llvm module passes [0]
  time: 26.816; rss: 625MB      codegen passes [0]
  time: 0.003; rss: 625MB       codegen passes [0]
time: 126.276; rss: 625MB       LLVM passes

-Z orbit:

time: 12.615; rss: 727MB        translation
  time: 4.449; rss: 634MB       llvm function passes [0]
  time: 69.080; rss: 634MB      llvm module passes [0]
  time: 25.928; rss: 636MB      codegen passes [0]
  time: 0.002; rss: 604MB       codegen passes [0]
time: 100.045; rss: 604MB       LLVM passes

LLVM spends 26 seconds less optimizing the result of trans and uses 20 MBs less of RAM, at the cost of one extra second spent in MIR translation (which could be some missing caching on type/trait stuff).

[sanxiyn]

Will old trans be tested by Buildbot after this PR is merged?

[eddyb]

@sanxiyn Good question. We could technically flip the MIR bots to be "anti-MIR". cc @alexcrichton

[eddyb]

@alexcrichton As explained in the PR description, both RUSTFLAGS="-Zorbit=off" and the #[rustc_no_mir] function attribute will work to switch to old trans for the whole compilation or just a function.

[alexcrichton]

Clearly I also need a PR to read more things, thanks @eddyb!

[alexcrichton]

I'm going to close this just to help clear out the queue, but the progress for this is tracked on https://github.com/rust-lang/rust/milestone/29

[nikomatsakis]

After some discussion on IRC, we decided to go forward with this -- MIR is looking good, with no known functional regressions, and we're ready to see how it fares with the broader, nightly audience. =) As @eddyb wrote, albeit for a different nightly:

I would like this PR to get into nightly soon so that we can get early feedback in this release cycle and focus on correctness fixes and performance improvements, with the potential for removing the old backend implementation before beta branches off.

I think that last part may be a bit optimistic, but then... here's hoping. =)

[nikomatsakis]

@bors r+

[bors]

📌 Commit d2f2df1 has been approved by nikomatsakis

[bors]

⌛️ Testing commit d2f2df1 with merge 5713448...

[bors]

💔 Test failed - auto-mac-32-opt

[eddyb]

@bors r=nikomatsakis

[bors]

📌 Commit b583711 has been approved by nikomatsakis

[bors]

⌛️ Testing commit b583711 with merge 34d14e7...

[bors]

☀️ Test successful - auto-linux-32-nopt-t, auto-linux-32-opt, auto-linux-64-cargotest, auto-linux-64-cross-freebsd, auto-linux-64-debug-opt, auto-linux-64-nopt-t, auto-linux-64-opt, auto-linux-64-opt-mir, auto-linux-64-opt-rustbuild, auto-linux-64-x-android-t, auto-linux-cross-opt, auto-linux-musl-64-opt, auto-mac-32-opt, auto-mac-64-opt, auto-mac-64-opt-rustbuild, auto-mac-cross-ios-opt, auto-win-gnu-32-opt, auto-win-gnu-32-opt-rustbuild, auto-win-gnu-64-opt, auto-win-msvc-32-opt, auto-win-msvc-64-cargotest, auto-win-msvc-64-opt, auto-win-msvc-64-opt-mir, auto-win-msvc-64-opt-rustbuild
Approved by: nikomatsakis
Pushing 34d14e7 to master...