New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rustc: Remove `dylib` crate type from most rustc crates #56987

Open
wants to merge 1 commit into
base: master
from

Conversation

Projects
None yet
@alexcrichton
Copy link
Member

alexcrichton commented Dec 19, 2018

Now that procedural macros no longer link transitively to libsyntax,
this shouldn't be needed any more! This commit is an experiment in
removing all dynamic libraries from rustc except for librustc_driver
itself. Let's see how far we can get with that!

More details below about this change

@rust-highfive

This comment has been minimized.

Copy link
Collaborator

rust-highfive commented Dec 19, 2018

r? @pnkfelix

(rust_highfive has picked a reviewer for you, use r? to override)

@alexcrichton

This comment has been minimized.

Copy link
Member

alexcrichton commented Dec 19, 2018

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Dec 19, 2018

⌛️ Trying commit 6419362 with merge b7ced6c...

bors added a commit that referenced this pull request Dec 19, 2018

Auto merge of #56987 - alexcrichton:less-dylibs, r=<try>
rustc: Remove `dylib` crate type from most rustc crates

Now that procedural macros no longer link transitively to libsyntax,
this shouldn't be needed any more! This commit is an experiment in
removing all dynamic libraries from rustc except for librustc_driver
itself. Let's see how far we can get with that!
@Mark-Simulacrum

This comment has been minimized.

Copy link
Member

Mark-Simulacrum commented Dec 19, 2018

I believe this won't work - clippy, codegen, perhaps miri aren't built in the same target directory as librustc_* so without dylib I think they won't be able to load those libraries from the prelude? Perhaps I'm forgetting what files we copy/need though and just the rlib is enough?

Though in that case it's not super clear to me why proc macros needed the dylibs either...

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Dec 19, 2018

☀️ Test successful - status-travis
State: approved= try=True

@alexcrichton

This comment has been minimized.

Copy link
Member

alexcrichton commented Dec 19, 2018

@Mark-Simulacrum heh well I can empirically say that it does indeed work! All of those tools load the compiler primarily through the rustc_driver crate which will now statically include all the other compiler crates. The target directory already isn't shared with rustc, and they all load compiler crates through the sysroot anyway.

Any compiler crate can be loaded at any time through the sysroot, just because they aren't dylibs doesn't mean they can't be loaded (you can already today load the compiler's own crates.io dependencies). The sysroot still contains all the intermediate rlib files.

Locally I've run ./x.py test src/tools/{clippy,rls} as well as ./x.py build src/tools/miri (miri is currently test-fail on master). Getting everything working only required one expected diff against clippy:

diff --git a/src/lib.rs b/src/lib.rs
index 40694726..5cb1a737 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -17,6 +17,8 @@
 // (currently there is no way to opt into sysroot crates w/o `extern crate`)
 #[allow(unused_extern_crates)]
 extern crate rustc_plugin;
+#[allow(unused_extern_crates)]
+extern crate rustc_driver;
 use self::rustc_plugin::Registry;
 
 #[plugin_registrar]
@alexcrichton

This comment has been minimized.

Copy link
Member

alexcrichton commented Dec 19, 2018

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Dec 19, 2018

Success: Queued b7ced6c with parent 6f839fb, comparison URL.

alexcrichton added a commit to alexcrichton/rust-clippy that referenced this pull request Dec 19, 2018

Link to `rustc_driver` crate in plugin
This is in anticipation for rust-lang/rust#56987 where the
`rustc_driver` crate being linked in will be required to link correctly
against the compiler. In the meantime it should be harmless otherwise!
@Mark-Simulacrum

This comment has been minimized.

Copy link
Member

Mark-Simulacrum commented Dec 19, 2018

Okay, yeah, that makes sense. I guess I just didn't quite connect the dots so to speak on rlib crates being loadable, but of course they have to be as otherwise the whole ecosystem wouldn't work.


However, one additional concern with doing this is that it bumps artifact sizes by ~1.6x (from 487 MB to 765 MB). With a rustup-toolchain-install-master on the master and try commits from the try build, I see the following "top 10" file sizes. I suspect that this fairly sizeable growth may be quite painful for downstream users; we already ship nearly ~1 GB of data across all of our tools -- this feels like it would increase that by 250 MB. That's quite a lot, in general, though we can probably afford it. I suspect that MIR-only rlibs would help here but we probably can't "just" do that at this point.

$ find master -type f | xargs du -ah | sort -rh | head
74M     master/lib/rustlib/x86_64-unknown-linux-gnu/codegen-backends/librustc_codegen_llvm-llvm.so
68M     master/lib/rustlib/x86_64-unknown-linux-gnu/lib/libLLVM-8svn.so
36M     master/lib/rustlib/x86_64-unknown-linux-gnu/codegen-backends/librustc_codegen_llvm-emscripten.so
36M     master/bin/rustdoc
26M     master/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc-8197a5f727130e18.so
26M     master/lib/librustc-8197a5f727130e18.so
25M     master/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-f929d44e8ee822a6.rlib
11M     master/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_mir-9e333f7280aadc85.so
11M     master/lib/librustc_mir-9e333f7280aadc85.so
9.9M    master/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-317bdea014a4d323.rlib

$ find try -type f | xargs du -ah | sort -rh | head
116M    try/lib/rustlib/x86_64-unknown-linux-gnu/codegen-backends/librustc_codegen_llvm-llvm.so
87M     try/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc-749452a0b8d67c66.rlib
78M     try/lib/rustlib/x86_64-unknown-linux-gnu/codegen-backends/librustc_codegen_llvm-emscripten.so
68M     try/lib/rustlib/x86_64-unknown-linux-gnu/lib/libLLVM-8svn.so
57M     try/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_driver-7422448280a926b8.so
57M     try/lib/librustc_driver-7422448280a926b8.so
37M     try/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_mir-5bcf33ecc7db40f1.rlib
36M     try/bin/rustdoc
25M     try/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-8a01f40ec9cf63ef.rlib
23M     try/lib/rustlib/x86_64-unknown-linux-gnu/lib/libsyntax-c233c5cd5247336c.rlib
@alexcrichton

This comment has been minimized.

Copy link
Member

alexcrichton commented Dec 19, 2018

That's a good point! I hadn't written up a full summary for this yet because if it doesn't affect perf at all it's probably not worth it. That being said it's definitely expected that this will increase sizes. Rust dylibs are naturally much smaller than rlibs for two reasons:

  • First, metadata is compressed in a dylib
  • Second, the object code for all rust crates is now present twice in the sysroot, once in the librustc_driver.so dylib and once in the crate's own rlib (like librustc.rlib).

That's just to say there's no hacks here AFAIK, it's just fundamental that the rlib sizes will be much larger.

I don't think installation size is really all that important, though. So long as we're not an order of magnitude off at least! The more imporant metric (I think at least) is the download size. Downloading the rust-std+rustc components before this commit was 142MB, and this commit (from try) is 232 megabytes. Funnily enough that's also a 1.6x increase!

I sort of highly doubt that anyone will notice this in practice. We add dependencies to the compiler willy-nilly and have already increased our download size 20% over the last year with binaries alone (that's not even taking into account the 86% increase over the last year in rust-docs download size).

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Dec 19, 2018

Finished benchmarking try commit b7ced6c

@mati865

This comment has been minimized.

Copy link
Contributor

mati865 commented Dec 19, 2018

Impressive amount of green 🎉

alexcrichton added a commit to alexcrichton/rust-clippy that referenced this pull request Dec 19, 2018

Link to `rustc_driver` crate in plugin
This is in anticipation for rust-lang/rust#56987 where the
`rustc_driver` crate being linked in will be required to link correctly
against the compiler. In the meantime it should be harmless otherwise!

@alexcrichton alexcrichton force-pushed the alexcrichton:less-dylibs branch from 6419362 to b503a94 Dec 19, 2018

@alexcrichton

This comment has been minimized.

Copy link
Member

alexcrichton commented Dec 19, 2018

Ok, great! I think that's green enough that this is worth considering at least. To detail this a bit more, we've historically used the dylib crate type for all rustc crates. That means all rustc crates are shipped as dynamic libraries, dynamically linking to one another. As to why we've done this for so long, I don't really know why. My best guess would be that when the compiler was implemented only dynamic libraries were supported. By the time Cargo came along to build rustc (after 1.0 was released) and opened up easily producing rlibs, we I think just never really got around to changing how things were done.

Note that today tons of rlibs ship with the compiler. All of rustc's crates.io dependencies are compiled as rlibs and linked into one dylib or another. What this PR is doing is basically creating one dynamic library for the compiler, only librustc_driver.so. Everything else is now compiled to an rlib instead of a dylib, and then everything is linked statically into one large librustc_driver.so. This notably means that we're shipping rlibs of compiler dependencies instead of dylibs, so we can't stop shipping those.

Any tool which depends on the compiler (RLS, miri, clippy, rustdoc, etc) all use rustc through the rustc_driver crate pretty much already. You can still access any other crate, but linking works best if rustc_driver is linked as well (even if it's not used). This had to be added to clippy for example, but all other tools work as-is (the RLS even worked with the previous try build!).

There's a few major consequences as a result from this change:

  • First, we have way fewer dynamic libraires in rustc's execution. This in turn means there's way less work to be done in terms of dynamic relocations because so much of rustc's inter-library calls are now static. This is where all the perf improvements are coming from. Note, however, that I believe these perf improvements are Linux-specific (and may affect OSX?). I believe I've measured in the past and a change like this doesn't have much effect on Windows. Keep in mind that all the green from the perf run is mostly tens of milliseconds here and there. It's definitely less work for the dynamic loader to do, but no one's likely to really feel the impact of a change like this.

  • Next, we're shipping rlibs instead of dylibs. This means that we're not only shipping uncompressed metadata for all of these libraries but we're also shipping double the object code (once in the rlib and once as it's linked into librustc_driver.so). There's some discussion above about the various size changes happening here.

  • Finally we're able to remove the rustc_cratesio_shim crate from the compiler. That's just a "linking hack" needed and is now obviated because everything isn't linked until the very end with librustc_driver.so.

I'm curious what others think of this change! This ideally shouldn't affect how the compiler is really developed that much, but I think I'd like to land this for the dynamic loader improvements on Linux. The dynamic library surely still has an absolutely massive symbol table, but it'd be great if we could try to minimize it in the future via one way or another. My favorite part about this, however, is how it simplifies the distribution of the compiler as there's only one dynamic library instead of a large proliferating set!

Also note that the rustc binary still has a dependency on a few other dynamic libraries, notably libstd.so, libterm.so, and libtest.so (term comes from test). I'd like to remove these as well in the long run, only shipping one self-contained librustc_driver.so, but that's a change for another day! (and unlikely to bring about any wins).

Ok for a randomly selected compiler team member...

r? @eddyb

and also cc @rust-lang/compiler and @rust-lang/release, but feel free to unsubscribe yourself if you're not interested in this change

@rust-highfive rust-highfive assigned eddyb and unassigned alexcrichton Dec 19, 2018

@Zoxc

This comment has been minimized.

Copy link
Contributor

Zoxc commented Dec 19, 2018

I'm definitely in favor of this change. This should also speed up tests on Linux and enable us to throw ThinLTO on librustc_driver.so, which will effectively be the whole compiler.

bors added a commit to rust-lang/rust-clippy that referenced this pull request Dec 20, 2018

Auto merge of #3564 - alexcrichton:rustc-driver, r=phansch
Link to `rustc_driver` crate in plugin

This is in anticipation for rust-lang/rust#56987 where the
`rustc_driver` crate being linked in will be required to link correctly
against the compiler. In the meantime it should be harmless otherwise!
@michaelwoerister

This comment has been minimized.

Copy link
Contributor

michaelwoerister commented Dec 20, 2018

Cool! Thanks, Alex!

I'd also be interested whether we can do some kind of cross-crate LTO in the future.

@nagisa

This comment has been minimized.

Copy link
Contributor

nagisa commented Dec 20, 2018

Next, we're shipping rlibs instead of dylibs. This means that we're not only shipping uncompressed metadata for all of these libraries but we're also shipping double the object code (once in the rlib and once as it's linked into librustc_driver.so). There's some discussion above about the various size changes happening here.

Might be interesting to add -Zmetadata=compressed that would make the rlib metadata compressed regardless, but that should not affect the download size much as the tarball compresses the metadata one way or the other... but I’d much rather slightly slower compilation when linking in the compiler crates (rare) over keeping extra megabytes on my disk (adds up for all the toolchains I tend to keep around).

It hurts somewhat that we are increasing the download size by 1.6x here. Just 25% decrease was a major drive in adding xz tarballs in the first place... It makes me wonder whether it is really necessary to have all the compiler crates as rlibs at all. For example I cannot imagine anybody needing rustc_data_structures, rustc_arena or rustc_apfloat in the sysroot for any reason. It would make sense to me to just ship rustc_driver and re-export all that tools need from it, perhaps even whole crates (pub extern crate ... instead of extern crate ...).

@alexcrichton

This comment has been minimized.

Copy link
Member

alexcrichton commented Dec 20, 2018

@Zoxc @michaelwoerister ah yes good points! While we're not really ready to just flip a switch to do that, this would be a prerequisite for doing it.

@rust-highfive

This comment has been minimized.

Copy link
Collaborator

rust-highfive commented Jan 8, 2019

The job x86_64-apple of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
[00:02:28]       Memory: 8 GB
[00:02:28]       Boot ROM Version: VMW71.00V.0.B64.1704110547
[00:02:28]       Apple ROM Info: [MS_VM_CERT/SHA1/27d66596a61c48dd3dc7216fd715126e33f59ae7]Welcome to the Virtual Machine
[00:02:28]       SMC Version (system): 2.8f0
[00:02:28]       Serial Number (system): VMT60qCqQgJN
[00:02:28] 
[00:02:28] hw.ncpu: 4
[00:02:28] hw.byteorder: 1234
[00:02:28] hw.memsize: 8589934592
---
[00:29:19] [TIMING] Assemble { target_compiler: Compiler { stage: 1, host: "x86_64-apple-darwin" } } -- 0.253
[00:29:19] travis_fold:start:stage1-std
travis_time:start:stage1-std
Building stage1 std artifacts (x86_64-apple-darwin -> x86_64-apple-darwin)
[00:29:20] error: failed to run `rustc` to learn about target-specific information
[00:29:20] Caused by:
[00:29:20] Caused by:
[00:29:20]   process didn't exit successfully: `/Users/travis/build/rust-lang/rust/build/bootstrap/debug/rustc - --crate-name ___ --print=file-names --target x86_64-apple-darwin --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro` (exit code: 101)
[00:29:20] --- stderr
[00:29:20] thread 'rustc' panicked at 'cannot access a scoped thread local variable without calling `set` first', /Users/travis/.cargo/registry/src/github.com-1ecc6299db9ec823/scoped-tls-0.1.2/src/lib.rs:186:9
[00:29:20] 
[00:29:20] error: internal compiler error: unexpected panic
[00:29:20] 
[00:29:20] note: the compiler unexpectedly panicked. this is a bug.
[00:29:20] note: the compiler unexpectedly panicked. this is a bug.
[00:29:20] 
[00:29:20] note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports
[00:29:20] 
[00:29:20] note: rustc 1.33.0-dev running on x86_64-apple-darwin
[00:29:20] 
[00:29:20] note: compiler flags: -Z osx-rpath-install-name -Z force-unstable-if-unmarked -C prefer-dynamic -C debug-assertions=n -C codegen-units=1 -C link-args=-Wl,-rpath,@loader_path/../lib --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro
[00:29:20] 
[00:29:20] [RUSTC-TIMING] ___ test:false 0.057
[00:29:20] command did not execute successfully: "/Users/travis/build/rust-lang/rust/build/x86_64-apple-darwin/stage0/bin/cargo" "build" "--target" "x86_64-apple-darwin" "-j" "4" "--release" "--locked" "--color" "always" "--features" "panic-unwind backtrace profiler" "--manifest-path" "/Users/travis/build/rust-lang/rust/src/libstd/Cargo.toml" "--message-format" "json"
[00:29:20] expected success, got: exit code: 101
[00:29:20] failed to run: /Users/travis/build/rust-lang/rust/build/bootstrap/debug/bootstrap build
[00:29:20] Build completed unsuccessfully in 0:25:50
[00:29:20] Build completed unsuccessfully in 0:25:50
[00:29:20] make: *** [all] Error 1
The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:02f037f3
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
Tue Jan  8 11:28:54 GMT 2019
---
travis_fold:start:after_failure.2
travis_time:start:1a961886
$ ls -lat $HOME/Library/Logs/DiagnosticReports/
total 0
drwx------+ 15 travis  staff  510 Jan 25  2018 ..
drwx------   2 travis  staff   68 Dec  6  2017 .
travis_fold:end:after_failure.2
travis_fold:start:after_failure.3
travis_time:start:24e3e320
$ find $HOME/Library/Logs/DiagnosticReports -type f -name '*.crash' -not -name '*.stage2-*.crash' -not -name 'com.apple.CoreSimulator.CoreSimulatorService-*.crash' -exec printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" {} \; -exec head -750 {} \; -exec echo travis_fold":"end:crashlog \; || true
$ find $HOME/Library/Logs/DiagnosticReports -type f -name '*.crash' -not -name '*.stage2-*.crash' -not -name 'com.apple.CoreSimulator.CoreSimulatorService-*.crash' -exec printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" {} \; -exec head -750 {} \; -exec echo travis_fold":"end:crashlog \; || true
travis_time:end:24e3e320:start=1546946937282196000,finish=1546946937297423000,duration=15227000
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:039fe484
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:11753793
travis_time:start:11753793
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:19c092b1
$ dmesg | grep -i kill
$ dmesg | grep -i kill
Unable to obtain kernel buffer: Operation not permitted
usage: sudo dmesg
travis_fold:end:after_failure.6

Done. Your build exited with 1.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@alexcrichton

This comment has been minimized.

Copy link
Member

alexcrichton commented Jan 8, 2019

@petrochenkov I think it's a great idea and would be in favor of going that direction. We do not have the capability to do it today as-is, though, and such a change would require a number of compiler changes.

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Jan 8, 2019

☔️ The latest upstream changes (presumably #57429) made this pull request unmergeable. Please resolve the merge conflicts.

@nagisa

This comment has been minimized.

Copy link
Contributor

nagisa commented Jan 8, 2019

We do not have the capability to do it today as-is, though, and such a change would require a number of compiler changes.

What do you mean exactly? Would those changes be internal to how the compiler is built or is it rather that the compiler is missing a feature that prevents the strategy altogether somehow?

@alexcrichton alexcrichton force-pushed the alexcrichton:less-dylibs branch from 4f49c8f to 6eee8e0 Jan 8, 2019

@alexcrichton

This comment has been minimized.

Copy link
Member

alexcrichton commented Jan 8, 2019

@nagisa if we don't ship rlibs and/or we only ship metadata then extern crate rustc_driver doesn't work because the compiler will look for all the various dependencies, none of which will exist. We'd need to add support to rustc to load metadata for foreign crates from the current local crate.

@bors: r=michaelwoerister

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Jan 8, 2019

📌 Commit 6eee8e0 has been approved by michaelwoerister

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Jan 9, 2019

⌛️ Testing commit 6eee8e0 with merge cf06026...

bors added a commit that referenced this pull request Jan 9, 2019

Auto merge of #56987 - alexcrichton:less-dylibs, r=michaelwoerister
rustc: Remove `dylib` crate type from most rustc crates

Now that procedural macros no longer link transitively to libsyntax,
this shouldn't be needed any more! This commit is an experiment in
removing all dynamic libraries from rustc except for librustc_driver
itself. Let's see how far we can get with that!

[More details below about this change](#56987 (comment))
@bors

This comment has been minimized.

Copy link
Contributor

bors commented Jan 9, 2019

💔 Test failed - status-travis

@rust-highfive

This comment has been minimized.

Copy link
Collaborator

rust-highfive commented Jan 9, 2019

The job x86_64-apple of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
[00:02:26]       Memory: 8 GB
[00:02:26]       Boot ROM Version: VMW71.00V.0.B64.1704110547
[00:02:26]       Apple ROM Info: [MS_VM_CERT/SHA1/27d66596a61c48dd3dc7216fd715126e33f59ae7]Welcome to the Virtual Machine
[00:02:26]       SMC Version (system): 2.8f0
[00:02:26]       Serial Number (system): VMlNyEWzlNxZ
[00:02:26] 
[00:02:26] hw.ncpu: 4
[00:02:26] hw.byteorder: 1234
[00:02:26] hw.memsize: 8589934592
---
[00:29:25] [TIMING] Assemble { target_compiler: Compiler { stage: 1, host: "x86_64-apple-darwin" } } -- 0.247
[00:29:25] travis_fold:start:stage1-std
travis_time:start:stage1-std
Building stage1 std artifacts (x86_64-apple-darwin -> x86_64-apple-darwin)
[00:29:26] error: failed to run `rustc` to learn about target-specific information
[00:29:26] Caused by:
[00:29:26] Caused by:
[00:29:26]   process didn't exit successfully: `/Users/travis/build/rust-lang/rust/build/bootstrap/debug/rustc - --crate-name ___ --print=file-names --target x86_64-apple-darwin --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro` (exit code: 101)
[00:29:26] --- stderr
[00:29:26] thread 'rustc' panicked at 'cannot access a scoped thread local variable without calling `set` first', /Users/travis/.cargo/registry/src/github.com-1ecc6299db9ec823/scoped-tls-0.1.2/src/lib.rs:186:9
[00:29:26] 
[00:29:26] error: internal compiler error: unexpected panic
[00:29:26] 
[00:29:26] note: the compiler unexpectedly panicked. this is a bug.
[00:29:26] note: the compiler unexpectedly panicked. this is a bug.
[00:29:26] 
[00:29:26] note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports
[00:29:26] 
[00:29:26] note: rustc 1.33.0-dev running on x86_64-apple-darwin
[00:29:26] 
[00:29:26] note: compiler flags: -Z osx-rpath-install-name -Z force-unstable-if-unmarked -C prefer-dynamic -C debug-assertions=n -C codegen-units=1 -C link-args=-Wl,-rpath,@loader_path/../lib --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro
[00:29:26] 
[00:29:26] [RUSTC-TIMING] ___ test:false 0.057
[00:29:26] command did not execute successfully: "/Users/travis/build/rust-lang/rust/build/x86_64-apple-darwin/stage0/bin/cargo" "build" "--target" "x86_64-apple-darwin" "-j" "4" "--release" "--locked" "--color" "always" "--features" "panic-unwind backtrace profiler" "--manifest-path" "/Users/travis/build/rust-lang/rust/src/libstd/Cargo.toml" "--message-format" "json"
[00:29:26] expected success, got: exit code: 101
[00:29:26] failed to run: /Users/travis/build/rust-lang/rust/build/bootstrap/debug/bootstrap build
[00:29:26] Build completed unsuccessfully in 0:26:01
[00:29:26] Build completed unsuccessfully in 0:26:01
[00:29:26] make: *** [all] Error 1
The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:09fce50a
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
Wed Jan  9 01:39:37 GMT 2019
---
travis_fold:start:after_failure.2
travis_time:start:078ccee8
$ ls -lat $HOME/Library/Logs/DiagnosticReports/
total 0
drwx------+ 15 travis  staff  510 Jan 25  2018 ..
drwx------   2 travis  staff   68 Dec  6  2017 .
travis_fold:end:after_failure.2
travis_fold:start:after_failure.3
travis_time:start:0d716850
$ find $HOME/Library/Logs/DiagnosticReports -type f -name '*.crash' -not -name '*.stage2-*.crash' -not -name 'com.apple.CoreSimulator.CoreSimulatorService-*.crash' -exec printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" {} \; -exec head -750 {} \; -exec echo travis_fold":"end:crashlog \; || true
$ find $HOME/Library/Logs/DiagnosticReports -type f -name '*.crash' -not -name '*.stage2-*.crash' -not -name 'com.apple.CoreSimulator.CoreSimulatorService-*.crash' -exec printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" {} \; -exec head -750 {} \; -exec echo travis_fold":"end:crashlog \; || true
travis_time:end:0d716850:start=1546997979861768000,finish=1546997979880216000,duration=18448000
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:14a68b74
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:03251c5b
travis_time:start:03251c5b
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:0706be9a
$ dmesg | grep -i kill
$ dmesg | grep -i kill
Unable to obtain kernel buffer: Operation not permitted
usage: sudo dmesg
travis_fold:end:after_failure.6

Done. Your build exited with 1.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

rustc: Remove `dylib` crate type from most rustc crates
Now that procedural macros no longer link transitively to libsyntax,
this shouldn't be needed any more! This commit is an experiment in
removing all dynamic libraries from rustc except for librustc_driver
itself. Let's see how far we can get with that!

@alexcrichton alexcrichton force-pushed the alexcrichton:less-dylibs branch from 6eee8e0 to 36a185e Jan 9, 2019

@alexcrichton

This comment has been minimized.

Copy link
Member

alexcrichton commented Jan 9, 2019

@bors: r=michaelwoerister

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Jan 9, 2019

📌 Commit 36a185e has been approved by michaelwoerister

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Jan 9, 2019

⌛️ Testing commit 36a185e with merge faebe86...

bors added a commit that referenced this pull request Jan 9, 2019

Auto merge of #56987 - alexcrichton:less-dylibs, r=michaelwoerister
rustc: Remove `dylib` crate type from most rustc crates

Now that procedural macros no longer link transitively to libsyntax,
this shouldn't be needed any more! This commit is an experiment in
removing all dynamic libraries from rustc except for librustc_driver
itself. Let's see how far we can get with that!

[More details below about this change](#56987 (comment))
@bors

This comment has been minimized.

Copy link
Contributor

bors commented Jan 9, 2019

💔 Test failed - status-appveyor

@mati865

This comment has been minimized.

Copy link
Contributor

mati865 commented Jan 9, 2019

Memory access violation:

Building stage1 std artifacts (x86_64-pc-windows-gnu -> x86_64-pc-windows-gnu)
error: failed to run `rustc` to learn about target-specific information
Caused by:
 process didn't exit successfully: `C:\projects\rust\build\bootstrap/debug/rustc - --crate-name ___ --print=file-names --target x86_64-pc-windows-gnu --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro` (exit code: 3221225477)
@bors

This comment has been minimized.

Copy link
Contributor

bors commented Jan 16, 2019

☔️ The latest upstream changes (presumably #57416) made this pull request unmergeable. Please resolve the merge conflicts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment