New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple codegen units breaks linking: "undefined reference to rust_begin_unwind" #47074

Closed
japaric opened this Issue Dec 30, 2017 · 8 comments

Comments

Projects
None yet
2 participants
@japaric
Member

japaric commented Dec 30, 2017

STR

$ git clone https://github.com/clebi/stm32_magnetic

$ cd stm32_magnetic

$ git checkout 086b7d9deb1ae7bd3d82d63f65eb9430fab677c1

$ # disable incremental compilation to avoid other issues
$ echo '[profile.dev]' >> Cargo.toml
$ echo 'incremental = false' >> Cargo.toml

$ # discard the Xargo sysroot to avoid problems with stale artifacts
$ rm -rf ~/.xargo

$ # this uses multiple codegen units
$ xargo rustc -- -Z print-link-args > link-args
(..)
  = note: /home/japaric/.xargo/lib/rustlib/thumbv7em-none-eabihf/lib/libcore-82bb515fb6a7673a.rlib(core-82bb515fb6a7673a.core13-a2b2f175d842448d578bfd1df50c2091.rs.rcgu.o): In function `core::panicking::panic_fmt':
          /home/japaric/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/panicking.rs:71: undefined reference to `rust_begin_unwind'

$ cat link-args | tr ' ' '\n'
"arm-none-eabi-ld"
"-L"
"/home/japaric/.xargo/lib/rustlib/thumbv7em-none-eabihf/lib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic0.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic1.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic10.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic11.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic12.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic13.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic14.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic15.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic2.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic3.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic4.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic5.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic6.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic7.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic8.rcgu.o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d.i2c_magnetic9.rcgu.o"
"-o"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/i2c_magnetic-da54b67a466eab3d"
"--gc-sections"
"-L"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps"
"-L"
"/home/japaric/tmp/stm32_magnetic/target/debug/deps"
"-L"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/build/cortex-m-rt-67ae64ff3b026a1b/out"
"-L"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/build/f3-9427da8830940471/out"
"-L"
"/home/japaric/.xargo/lib/rustlib/thumbv7em-none-eabihf/lib"
"-Bstatic"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libcortex_m_rtfm-c0937f58fbde0459.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/librtfm_core-1e85456d9189c8f2.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libuntagged_option-cb0ee6b31e8b5540.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libf3-5f30af66a432ec74.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libstm32f30x-5ab99d65170ba613.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libcortex_m_rt-a656f43a1b217a17.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libr0-eea37c5929500808.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libstatic_ref-4e80cdb2ce553b2f.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libembedded_hal-a5c5f16836b1d4fa.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libcast-c3830481e6a0787e.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libnb-2c1e61e97adbfc28.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libcortex_m-8e97aa6b5ac0c1c3.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libbare_metal-c2470b1735ede45e.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libaligned-ca7b9812ff225cca.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libvolatile_register-a39da4476292d8c3.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libvcell-72540773165f79da.rlib"
"/home/japaric/tmp/stm32_magnetic/target/thumbv7em-none-eabihf/debug/deps/libcortex_m_semihosting-d2a306addcb7fb3a.rlib"
"/home/japaric/.xargo/lib/rustlib/thumbv7em-none-eabihf/lib/libcore-82bb515fb6a7673a.rlib"
"/home/japaric/.xargo/lib/rustlib/thumbv7em-none-eabihf/lib/libcompiler_builtins-21afa2c8d0d459e7.rlib"
"-Tlink.x"
"-Bdynamic"

$ # the linker command succeeds if the objects / archives are wrapped in --start-group / --end-group
$ "arm-none-eabi-ld" --start-group "-L" (..) "-Bdynamic" --end-group && echo OK
OK

$ # the build also succeeds if a single codegen unit is used
$ echo 'codegen-units = 1' >> Cargo.toml
$ rm -rf ~/.xargo && cargo clean
$ xargo build && echo OK
OK

Meta

$ rustc -V
rustc 1.24.0-nightly (77e189cd7 2017-12-28)

cc @alexcrichton @clebi

@japaric

This comment has been minimized.

Member

japaric commented Dec 30, 2017

After bisecting it seems that this broke between nightly-2017-12-27 (good) and nightly-2017-12-28 (bad).

@alexcrichton

This comment has been minimized.

Member

alexcrichton commented Dec 30, 2017

@japaric can you confirm where, for the embedded platform here, the symbol rust_begin_unwind is defined? I think this may be the same as #18807 (an ancient bug now!) where rustc doesn't understand this special symbol relationship well enough to pass the right linker flags.

@japaric

This comment has been minimized.

Member

japaric commented Dec 30, 2017

@alexcrichton the panic_fmt lang item, and several other lang items, is defined in the cortex-m-rt crate.

@alexcrichton

This comment has been minimized.

Member

alexcrichton commented Dec 31, 2017

Ok cool, makes sense! In that case I believe this is indeed the same issue as #18807.

The compiler knows who needs rust_begin_unwind (libcore) and also knows who defines it (libstd in general and cortex-m-rt in your case). It's the compiler's job to understand that this is a cyclic dependency and insert --start-group and --end-group appropriately but it currently does not.

I believe the correct fix here (and for #18807) is for rustc to detect which crate defines rust_begin_unwind and which crate requires it (std/core in general) and to automatically insert --start-group and --end-group around these libraries. That should be enough to make the linker error go away.

@japaric

This comment has been minimized.

Member

japaric commented Jan 11, 2018

ugh, I'm now seeing this linker error in another project when compiling it using the dev profile even though codegen-units is set to 1 and incremental is disabled in the Cargo.toml. My current fix is ... to enable LTO on the dev profile.

@alexcrichton any chance you'll have to time soon to implement the fix you mentioned? Otherwise I can look into it next week, probably.

@alexcrichton

This comment has been minimized.

Member

alexcrichton commented Jan 11, 2018

@japaric unfortunately I don't think I'll have time to look into this, but I can certainly help out with questions if they come up!

@japaric

This comment has been minimized.

Member

japaric commented Jan 11, 2018

@alexcrichton alright. I'll ping you on IRC when I get a chance to look into this.

dvc94ch added a commit to riscv-rust/riscv-crates that referenced this issue Feb 27, 2018

japaric added a commit to rust-embedded/discovery that referenced this issue Mar 2, 2018

bors bot added a commit to rust-embedded/discovery that referenced this issue Mar 2, 2018

Merge #70
70: work around rust-lang/rust#47074 r=japaric a=japaric

alexcrichton added a commit to alexcrichton/rust that referenced this issue Mar 23, 2018

rustc: Group linked libraries where needed
This commit fixes a longstanding issue with the compiler with circular
dependencies between libcore and libstd. The `core` crate requires at least one
symbol, the ability to unwind. The `std` crate is the crate which actually
defines this symbol, but the `std` crate also depends on the `core` crate.

This circular dependency is in general disallowed in Rust as crates cannot have
cycles amongst them. A special exception is made just for core/std, but this is
also unfortunately incompatible with how GNU linkers work. GNU linkers will
process undefined symbols in a left-to-right fashion, only actually linking an
rlib like libstd if there are any symbols used from it. This strategy is
incompatible with circular dependencies because if we otherwise don't use
symbols from libstd we don't discover that we needed it until we're later
processing libcore's symbols!

To fix this GNU linkers support the `--start-group` and `--end-group` options
which indicate "libraries between these markers may have circular dependencies
amongst them. The linker invocation has been updated to automatically pass these
arguments when we're invoking a GNU linker and automatically calculate where the
arguments need to go (around libstd and libcore)

Closes rust-lang#18807
Closes rust-lang#47074

alexcrichton added a commit to alexcrichton/rust that referenced this issue Mar 23, 2018

rustc: Group linked libraries where needed
This commit fixes a longstanding issue with the compiler with circular
dependencies between libcore and libstd. The `core` crate requires at least one
symbol, the ability to unwind. The `std` crate is the crate which actually
defines this symbol, but the `std` crate also depends on the `core` crate.

This circular dependency is in general disallowed in Rust as crates cannot have
cycles amongst them. A special exception is made just for core/std, but this is
also unfortunately incompatible with how GNU linkers work. GNU linkers will
process undefined symbols in a left-to-right fashion, only actually linking an
rlib like libstd if there are any symbols used from it. This strategy is
incompatible with circular dependencies because if we otherwise don't use
symbols from libstd we don't discover that we needed it until we're later
processing libcore's symbols!

To fix this GNU linkers support the `--start-group` and `--end-group` options
which indicate "libraries between these markers may have circular dependencies
amongst them. The linker invocation has been updated to automatically pass these
arguments when we're invoking a GNU linker and automatically calculate where the
arguments need to go (around libstd and libcore)

Closes rust-lang#18807
Closes rust-lang#47074
@alexcrichton

This comment has been minimized.

Member

alexcrichton commented Mar 23, 2018

I've opened a PR for this at #49316

alexcrichton added a commit to alexcrichton/rust that referenced this issue Mar 23, 2018

rustc: Group linked libraries where needed
This commit fixes a longstanding issue with the compiler with circular
dependencies between libcore and libstd. The `core` crate requires at least one
symbol, the ability to unwind. The `std` crate is the crate which actually
defines this symbol, but the `std` crate also depends on the `core` crate.

This circular dependency is in general disallowed in Rust as crates cannot have
cycles amongst them. A special exception is made just for core/std, but this is
also unfortunately incompatible with how GNU linkers work. GNU linkers will
process undefined symbols in a left-to-right fashion, only actually linking an
rlib like libstd if there are any symbols used from it. This strategy is
incompatible with circular dependencies because if we otherwise don't use
symbols from libstd we don't discover that we needed it until we're later
processing libcore's symbols!

To fix this GNU linkers support the `--start-group` and `--end-group` options
which indicate "libraries between these markers may have circular dependencies
amongst them. The linker invocation has been updated to automatically pass these
arguments when we're invoking a GNU linker and automatically calculate where the
arguments need to go (around libstd and libcore)

Closes rust-lang#18807
Closes rust-lang#47074

alexcrichton added a commit to alexcrichton/rust that referenced this issue Mar 25, 2018

rustc: Group linked libraries where needed
This commit fixes a longstanding issue with the compiler with circular
dependencies between libcore and libstd. The `core` crate requires at least one
symbol, the ability to unwind. The `std` crate is the crate which actually
defines this symbol, but the `std` crate also depends on the `core` crate.

This circular dependency is in general disallowed in Rust as crates cannot have
cycles amongst them. A special exception is made just for core/std, but this is
also unfortunately incompatible with how GNU linkers work. GNU linkers will
process undefined symbols in a left-to-right fashion, only actually linking an
rlib like libstd if there are any symbols used from it. This strategy is
incompatible with circular dependencies because if we otherwise don't use
symbols from libstd we don't discover that we needed it until we're later
processing libcore's symbols!

To fix this GNU linkers support the `--start-group` and `--end-group` options
which indicate "libraries between these markers may have circular dependencies
amongst them. The linker invocation has been updated to automatically pass these
arguments when we're invoking a GNU linker and automatically calculate where the
arguments need to go (around libstd and libcore)

Closes rust-lang#18807
Closes rust-lang#47074

alexcrichton added a commit to alexcrichton/rust that referenced this issue Mar 25, 2018

rustc: Group linked libraries where needed
This commit fixes a longstanding issue with the compiler with circular
dependencies between libcore and libstd. The `core` crate requires at least one
symbol, the ability to unwind. The `std` crate is the crate which actually
defines this symbol, but the `std` crate also depends on the `core` crate.

This circular dependency is in general disallowed in Rust as crates cannot have
cycles amongst them. A special exception is made just for core/std, but this is
also unfortunately incompatible with how GNU linkers work. GNU linkers will
process undefined symbols in a left-to-right fashion, only actually linking an
rlib like libstd if there are any symbols used from it. This strategy is
incompatible with circular dependencies because if we otherwise don't use
symbols from libstd we don't discover that we needed it until we're later
processing libcore's symbols!

To fix this GNU linkers support the `--start-group` and `--end-group` options
which indicate "libraries between these markers may have circular dependencies
amongst them. The linker invocation has been updated to automatically pass these
arguments when we're invoking a GNU linker and automatically calculate where the
arguments need to go (around libstd and libcore)

Closes rust-lang#18807
Closes rust-lang#47074

alexcrichton added a commit to alexcrichton/rust that referenced this issue Mar 27, 2018

rustc: Group linked libraries where needed
This commit fixes a longstanding issue with the compiler with circular
dependencies between libcore and libstd. The `core` crate requires at least one
symbol, the ability to unwind. The `std` crate is the crate which actually
defines this symbol, but the `std` crate also depends on the `core` crate.

This circular dependency is in general disallowed in Rust as crates cannot have
cycles amongst them. A special exception is made just for core/std, but this is
also unfortunately incompatible with how GNU linkers work. GNU linkers will
process undefined symbols in a left-to-right fashion, only actually linking an
rlib like libstd if there are any symbols used from it. This strategy is
incompatible with circular dependencies because if we otherwise don't use
symbols from libstd we don't discover that we needed it until we're later
processing libcore's symbols!

To fix this GNU linkers support the `--start-group` and `--end-group` options
which indicate "libraries between these markers may have circular dependencies
amongst them. The linker invocation has been updated to automatically pass these
arguments when we're invoking a GNU linker and automatically calculate where the
arguments need to go (around libstd and libcore)

Closes rust-lang#18807
Closes rust-lang#47074

alexcrichton added a commit to alexcrichton/rust that referenced this issue Mar 28, 2018

rustc: Group linked libraries where needed
This commit fixes a longstanding issue with the compiler with circular
dependencies between libcore and libstd. The `core` crate requires at least one
symbol, the ability to unwind. The `std` crate is the crate which actually
defines this symbol, but the `std` crate also depends on the `core` crate.

This circular dependency is in general disallowed in Rust as crates cannot have
cycles amongst them. A special exception is made just for core/std, but this is
also unfortunately incompatible with how GNU linkers work. GNU linkers will
process undefined symbols in a left-to-right fashion, only actually linking an
rlib like libstd if there are any symbols used from it. This strategy is
incompatible with circular dependencies because if we otherwise don't use
symbols from libstd we don't discover that we needed it until we're later
processing libcore's symbols!

To fix this GNU linkers support the `--start-group` and `--end-group` options
which indicate "libraries between these markers may have circular dependencies
amongst them. The linker invocation has been updated to automatically pass these
arguments when we're invoking a GNU linker and automatically calculate where the
arguments need to go (around libstd and libcore)

Closes rust-lang#18807
Closes rust-lang#47074

bors added a commit that referenced this issue Mar 29, 2018

Auto merge of #49316 - alexcrichton:start-group-end-group, r=michaelw…
…oerister

rustc: Group linked libraries where needed

This commit fixes a longstanding issue with the compiler with circular
dependencies between libcore and libstd. The `core` crate requires at least one
symbol, the ability to unwind. The `std` crate is the crate which actually
defines this symbol, but the `std` crate also depends on the `core` crate.

This circular dependency is in general disallowed in Rust as crates cannot have
cycles amongst them. A special exception is made just for core/std, but this is
also unfortunately incompatible with how GNU linkers work. GNU linkers will
process undefined symbols in a left-to-right fashion, only actually linking an
rlib like libstd if there are any symbols used from it. This strategy is
incompatible with circular dependencies because if we otherwise don't use
symbols from libstd we don't discover that we needed it until we're later
processing libcore's symbols!

To fix this GNU linkers support the `--start-group` and `--end-group` options
which indicate "libraries between these markers may have circular dependencies
amongst them. The linker invocation has been updated to automatically pass these
arguments when we're invoking a GNU linker and automatically calculate where the
arguments need to go (around libstd and libcore)

Closes #18807
Closes #47074

alexcrichton added a commit to alexcrichton/rust that referenced this issue Mar 29, 2018

rustc: Group linked libraries where needed
This commit fixes a longstanding issue with the compiler with circular
dependencies between libcore and libstd. The `core` crate requires at least one
symbol, the ability to unwind. The `std` crate is the crate which actually
defines this symbol, but the `std` crate also depends on the `core` crate.

This circular dependency is in general disallowed in Rust as crates cannot have
cycles amongst them. A special exception is made just for core/std, but this is
also unfortunately incompatible with how GNU linkers work. GNU linkers will
process undefined symbols in a left-to-right fashion, only actually linking an
rlib like libstd if there are any symbols used from it. This strategy is
incompatible with circular dependencies because if we otherwise don't use
symbols from libstd we don't discover that we needed it until we're later
processing libcore's symbols!

To fix this GNU linkers support the `--start-group` and `--end-group` options
which indicate "libraries between these markers may have circular dependencies
amongst them. The linker invocation has been updated to automatically pass these
arguments when we're invoking a GNU linker and automatically calculate where the
arguments need to go (around libstd and libcore)

Closes rust-lang#18807
Closes rust-lang#47074

bors added a commit that referenced this issue Mar 29, 2018

Auto merge of #49316 - alexcrichton:start-group-end-group, r=michaelw…
…oerister

rustc: Group linked libraries where needed

This commit fixes a longstanding issue with the compiler with circular
dependencies between libcore and libstd. The `core` crate requires at least one
symbol, the ability to unwind. The `std` crate is the crate which actually
defines this symbol, but the `std` crate also depends on the `core` crate.

This circular dependency is in general disallowed in Rust as crates cannot have
cycles amongst them. A special exception is made just for core/std, but this is
also unfortunately incompatible with how GNU linkers work. GNU linkers will
process undefined symbols in a left-to-right fashion, only actually linking an
rlib like libstd if there are any symbols used from it. This strategy is
incompatible with circular dependencies because if we otherwise don't use
symbols from libstd we don't discover that we needed it until we're later
processing libcore's symbols!

To fix this GNU linkers support the `--start-group` and `--end-group` options
which indicate "libraries between these markers may have circular dependencies
amongst them. The linker invocation has been updated to automatically pass these
arguments when we're invoking a GNU linker and automatically calculate where the
arguments need to go (around libstd and libcore)

Closes #18807
Closes #47074

@bors bors closed this in #49316 Mar 30, 2018

Aaronepower added a commit to Aaronepower/rust that referenced this issue May 1, 2018

rustc: Group linked libraries where needed
This commit fixes a longstanding issue with the compiler with circular
dependencies between libcore and libstd. The `core` crate requires at least one
symbol, the ability to unwind. The `std` crate is the crate which actually
defines this symbol, but the `std` crate also depends on the `core` crate.

This circular dependency is in general disallowed in Rust as crates cannot have
cycles amongst them. A special exception is made just for core/std, but this is
also unfortunately incompatible with how GNU linkers work. GNU linkers will
process undefined symbols in a left-to-right fashion, only actually linking an
rlib like libstd if there are any symbols used from it. This strategy is
incompatible with circular dependencies because if we otherwise don't use
symbols from libstd we don't discover that we needed it until we're later
processing libcore's symbols!

To fix this GNU linkers support the `--start-group` and `--end-group` options
which indicate "libraries between these markers may have circular dependencies
amongst them. The linker invocation has been updated to automatically pass these
arguments when we're invoking a GNU linker and automatically calculate where the
arguments need to go (around libstd and libcore)

Closes rust-lang#18807
Closes rust-lang#47074

berkus added a commit to metta-systems/vesper that referenced this issue Nov 10, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment