Recompiling causes panic #63349

mattrutherford · 2019-08-07T11:55:21Z

Making changes to code in modules that compile to WASM and running cargo build --release results in a panic.

This is the project: https://github.com/paritytech/substrate

Immediately running cargo clean && cargo build --release will allow successful compilation. Also upgrading to a new version of nightly will allow recompilation without cargo clean, but only the first recompile.

Meta

rustc --version --verbose:

rustc 1.38.0-nightly (6a91782 2019-08-06) running on x86_64-apple-darwin

Backtrace:

Fresh transaction-factory v0.0.1 (/Users/matt/projects/substrate/test-utils/transaction-factory)
error: failed to run custom build command for `node-runtime v2.0.0 (/Users/matt/projects/substrate/node/runtime)`

Caused by:
  process didn't exit successfully: `/Users/matt/projects/substrate/target/release/build/node-runtime-777af245baf20274/build-script-build` (exit code: 1)
--- stdout
Executing build command: "rustup" "run" "nightly" "cargo" "build" "--target=wasm32-unknown-unknown" "--manifest-path=/Users/matt/projects/substrate/target/release/wbuild/node-runtime/Cargo.toml" "--release"

--- stderr
   Compiling wasm-build-runner-impl v1.0.0 (/Users/matt/projects/substrate/target/release/build/node-runtime-3547c858aedc1441/out/wasm_build_runner)
    Finished release [optimized] target(s) in 0.25s
     Running `/Users/matt/projects/substrate/target/release/build/node-runtime-3547c858aedc1441/out/wasm_build_runner/target/release/wasm-build-runner-impl`
   Compiling node-runtime-wasm v1.0.0 (/Users/matt/projects/substrate/target/release/wbuild/node-runtime)
thread '<unnamed>' panicked at 'must have at least one serialized module', src/libcore/option.rs:1166:5
stack backtrace:
   0:        0x109957f12 - std::panicking::default_hook::{{closure}}::hbefba8356e31122b
   1:        0x109957bdd - std::panicking::default_hook::hb85a284cef21dddc
   2:        0x1085b55a3 - rustc::util::common::panic_hook::h8e13af8cece077e3
   3:        0x1099587d1 - std::panicking::rust_panic_with_hook::h8868b29516eb168e
   4:        0x10995822d - std::panicking::continue_panic_fmt::h8f85da70e3d42ed7
   5:        0x109958119 - rust_begin_unwind
   6:        0x109983e82 - core::panicking::panic_fmt::ha880b2fe6921e68e
   7:        0x109983f9e - core::option::expect_failed::hfea1a2f203365264
   8:        0x10bb851f8 - rustc_codegen_llvm::back::lto::run_fat::h0e6fc34b8aa41a0c
   9:        0x10bc04aa8 - rustc_codegen_ssa::back::write::start_executing_work::{{closure}}::h1c543b9ea0992759
  10:        0x10bc0b7d0 - std::sys_common::backtrace::__rust_begin_short_backtrace::h6b423e28f14e4e74
  11:        0x10bb5279c - std::panicking::try::do_call::h6fca1ea505f089a9
  12:        0x109967b4f - __rust_maybe_catch_panic
  13:        0x10bb52ba6 - core::ops::function::FnOnce::call_once{{vtable.shim}}::h6eb73fcc38df565a
  14:        0x10993bb1e - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h2a90a69cfe4e95fa
  15:        0x10996695e - std::sys::unix::thread::Thread::new::thread_start::h99953f013e830b09
  16:     0x7fff7c8392eb - _pthread_body
  17:     0x7fff7c83c249 - _pthread_start
query stack during panic:
end of query stack
thread 'rustc' panicked at 'src/librustc_codegen_ssa/back/write.rs:1826: panic during codegen/LLVM phase', src/librustc/util/bug.rs:37:26
stack backtrace:
   0:        0x109957f12 - std::panicking::default_hook::{{closure}}::hbefba8356e31122b
   1:        0x109957bdd - std::panicking::default_hook::hb85a284cef21dddc
   2:        0x1085b55a3 - rustc::util::common::panic_hook::h8e13af8cece077e3
   3:        0x1099587d1 - std::panicking::rust_panic_with_hook::h8868b29516eb168e
   4:        0x1085f9b25 - std::panicking::begin_panic::he8bde94d32f67181
   5:        0x10828fbb4 - rustc::util::bug::opt_span_bug_fmt::{{closure}}::h79b523f376a07816
   6:        0x10828f9ea - rustc::ty::context::tls::with_opt::{{closure}}::heef7fe9845124b11
   7:        0x10828f777 - rustc::ty::context::tls::with_context_opt::h61634ca6ab55ac5a
   8:        0x10828f7c2 - rustc::ty::context::tls::with_opt::h5ca54a02c4c0bb9b
   9:        0x10828faf8 - rustc::util::bug::opt_span_bug_fmt::h8f8a983ed3203bff
  10:        0x10828fa4b - rustc::util::bug::bug_fmt::h7fff681b74edf248
  11:        0x10bc09e05 - rustc_codegen_ssa::back::write::OngoingCodegen<B>::join::hd601e779fa22708f
  12:        0x10bc43d98 - <rustc_codegen_llvm::LlvmCodegenBackend as rustc_codegen_utils::codegen_backend::CodegenBackend>::join_codegen_and_link::hb5084914450219e0
  13:        0x1070296d2 - rustc_interface::queries::Query<T>::compute::h79dc64dd1d20cdc1
  14:        0x1070ede3c - rustc_interface::queries::<impl rustc_interface::interface::Compiler>::link::hc53a9984fbe3c770
  15:        0x106f5d224 - rustc_interface::interface::run_compiler_in_existing_thread_pool::hbcb4433730475c4a
  16:        0x106f90744 - std::thread::local::LocalKey<T>::with::hfc60cc57f753f9b0
  17:        0x106fa6228 - syntax::with_globals::h974a79eedc8d66fd
  18:        0x106f381a7 - std::sys_common::backtrace::__rust_begin_short_backtrace::ha8d874bf2a6ac95b
  19:        0x109967b4f - __rust_maybe_catch_panic
  20:        0x106f5eeb7 - core::ops::function::FnOnce::call_once{{vtable.shim}}::hf166e26f2620097b
  21:        0x10993bb1e - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h2a90a69cfe4e95fa
  22:        0x10996695e - std::sys::unix::thread::Thread::new::thread_start::h99953f013e830b09
  23:     0x7fff7c8392eb - _pthread_body
  24:     0x7fff7c83c249 - _pthread_start
query stack during panic:
end of query stack

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.38.0-nightly (6a91782b7 2019-08-06) running on x86_64-apple-darwin

note: compiler flags: -C opt-level=3 -C panic=abort -C lto -C incremental -C link-arg=--export-table -C link-arg=--export=__heap_base --crate-type cdylib

note: some of the compiler flags provided by cargo are hidden

error: Could not compile `node-runtime-wasm`.

The text was updated successfully, but these errors were encountered:

tesuji · 2019-08-07T11:59:35Z

Did you use sccache or anything similar?

mattrutherford · 2019-08-07T12:02:44Z

not locally where I'm experiencing the problem

nagisa · 2019-08-08T14:22:03Z

Assigning @michaelwoerister to investigate, there is very little else to do here.

michaelwoerister · 2019-08-09T12:09:41Z

I have not been able to reproduce this yet. @mattrutherford, would you mind providing the exact commands you are running and the set of source files you are modifying?

mattrutherford · 2019-08-13T19:41:36Z

yes, ok steps I can take to reproduce:

make a fresh clone of: https://github.com/paritytech/substrate
cargo build --release
Make change in some rust file in srml, such as: https://github.com/paritytech/substrate/blob/master/srml/support/src/dispatch.rs
cargo build --release

gives the error

hellow554 · 2019-08-19T07:04:44Z

@mattrutherford "Make change " can you specify it please? Where did you change what? A diff would be good (e.g. if you happen to have a cloned repo, then you can do git diff and show it to us).

michaelwoerister · 2019-08-19T11:52:51Z

I can reproduce on Fedora 30 when also setting CARGO_INCREMENTAL=1 in the environment. Thanks, @mattrutherford.

It's probably a buggy interaction incremental compilation and LTO (cc @alexcrichton). Not sure if it is wasm specific.

alexcrichton · 2019-08-19T14:16:12Z

Hm I was getting build errors locally and wasn't able to reproduce. I'm not really sure what compiler is necessary to fix errors like:

error[E0432]: unresolved import `alloc_crate::collections::CollectionAllocErr`
  --> /home/alex/.cargo/registry/src/github.com-1ecc6299db9ec823/hashmap_core-0.1.11/src/lib.rs:32:13
   |
32 |     pub use alloc_crate::collections::CollectionAllocErr;
   |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ no `CollectionAllocErr` in `collections`

   Compiling srml-support-procedural-tools v2.0.0 (/home/alex/code/substrate/srml/support/procedural/tools)

mattrutherford · 2019-08-23T12:44:35Z

Hm I was getting build errors locally and wasn't able to reproduce. I'm not really sure what compiler is necessary to fix errors like:

error[E0432]: unresolved import `alloc_crate::collections::CollectionAllocErr`
  --> /home/alex/.cargo/registry/src/github.com-1ecc6299db9ec823/hashmap_core-0.1.11/src/lib.rs:32:13
   |
32 |     pub use alloc_crate::collections::CollectionAllocErr;
   |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ no `CollectionAllocErr` in `collections`

   Compiling srml-support-procedural-tools v2.0.0 (/home/alex/code/substrate/srml/support/procedural/tools)

I think unfortunately you tried at a time where master was broken! If you can try again, maybe it's possible to reproduce the original problem

alexcrichton · 2019-08-27T17:03:16Z

Ok looks like this is actually pretty trivial to reproduce:

$ rustc +nightly foo.rs --crate-type cdylib -Copt-level=3 -C lto -C incremental=wut
$ rustc +nightly foo.rs --crate-type cdylib -Copt-level=3 -C lto -C incremental=wut
thread '<unnamed>' panicked at 'must have at least one serialized module', src/libcore/option.rs:1166:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
thread 'rustc' panicked at 'src/librustc_codegen_ssa/back/write.rs:1823: panic during codegen/LLVM phase', src/librustc/util/bug.rs:37:26

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.39.0-nightly (521d78407 2019-08-25) running on x86_64-unknown-linux-gnu

note: compiler flags: -C opt-level=3 -C lto -C incremental --crate-type cdylib

alexcrichton · 2019-08-27T19:31:16Z

Should be fixed by #63956

…michaelwoerister rustc: Handle modules in "fat" LTO more robustly When performing a "fat" LTO the compiler has a whole mess of codegen units that it links together. To do this it needs to select one module as a "base" module and then link everything else into this module. Previously LTO passes assume that there's at least one module in-memory to link into, but nowadays that's not always true! With incremental compilation modules may actually largely be cached and it may be possible that there's no in-memory modules to work with. This commit updates the logic of the LTO backend to handle modules a bit more uniformly during a fat LTO. This commit immediately splits them into two lists, one serialized and one in-memory. The in-memory list is then searched for the largest module and failing that we simply deserialize the first serialized module and link into that. This refactoring avoids juggling three lists, two of which are serialized modules and one of which is half serialized and half in-memory. Closes rust-lang#63349

jonas-schievink added A-incr-comp Area: Incremental compilation C-bug Category: This is a bug. I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ I-nominated T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Aug 7, 2019

mattrutherford changed the title ~~Re-compiling causes panic~~ Recompiling causes panic Aug 7, 2019

nagisa assigned michaelwoerister Aug 8, 2019

nagisa added P-high High priority and removed I-nominated labels Aug 8, 2019

Centril added the E-needs-mcve Call for participation: This issue has a repro, but needs a Minimal Complete and Verifiable Example label Aug 8, 2019

alexcrichton mentioned this issue Aug 27, 2019

rustc: Handle modules in "fat" LTO more robustly #63956

Merged

bors closed this as completed in 1a4330d Aug 29, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Recompiling causes panic #63349

Recompiling causes panic #63349

mattrutherford commented Aug 7, 2019

tesuji commented Aug 7, 2019

mattrutherford commented Aug 7, 2019

nagisa commented Aug 8, 2019

michaelwoerister commented Aug 9, 2019

mattrutherford commented Aug 13, 2019

hellow554 commented Aug 19, 2019

michaelwoerister commented Aug 19, 2019

alexcrichton commented Aug 19, 2019

mattrutherford commented Aug 23, 2019

alexcrichton commented Aug 27, 2019

alexcrichton commented Aug 27, 2019

Recompiling causes panic #63349

Recompiling causes panic #63349

Comments

mattrutherford commented Aug 7, 2019

Meta

tesuji commented Aug 7, 2019

mattrutherford commented Aug 7, 2019

nagisa commented Aug 8, 2019

michaelwoerister commented Aug 9, 2019

mattrutherford commented Aug 13, 2019

hellow554 commented Aug 19, 2019

michaelwoerister commented Aug 19, 2019

alexcrichton commented Aug 19, 2019

mattrutherford commented Aug 23, 2019

alexcrichton commented Aug 27, 2019

alexcrichton commented Aug 27, 2019