Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

error adding symbols: file format not recognized #8239

Closed
toku-sa-n opened this issue May 13, 2020 · 6 comments
Closed

error adding symbols: file format not recognized #8239

toku-sa-n opened this issue May 13, 2020 · 6 comments
Labels
A-linkage Area: linker issues, dylib, cdylib, shared libraries, so C-bug Category: bug Z-build-std Nightly: build-std

Comments

@toku-sa-n
Copy link

Problem

The file format of the output binary of cargo build -Zbuild-std=core,alloc is not recognized.

Steps

  1. Clone https://github.com/toku-sa-n/cargo_bug
  2. git checkout with_rlibc
  3. make

This causes the error: ld: target/cargo_settings/debug/librust_bug.a: error adding symbols: file format not recognized

Possible Solution(s)

Sorry, but I have no idea.

Notes

Output of cargo version:cargo 1.45.0-nightly (cb06cb2 2020-05-08)

rlibc is deprecated, but with compiler_builtins (which is the replacement of rlibc) , cargo build -Zbuild-std=core,alloc itself fails with the error: multiple rlib candidates for compiler_builtins found. rust-lang/wg-cargo-std-aware#53 seems to be related.

Building without compiler_builtins will cause another linker error: undefined reference to 'memcpy'.

Related issue: rust-osdev/cargo-xbuild#69

@toku-sa-n toku-sa-n added the C-bug Category: bug label May 13, 2020
@lights0123
Copy link

I have this issue on my custom target when specifying lto = true:

  = note: "nspire-gcc" "-L" "/home/benschattinger/Documents/Projects/Calculator/example-nspire/target/sysroot/lib/rustlib/armv5te-nspire-eabi/lib" "/home/benschattinger/Documents/Projects/Calculator/example-nspire/target/armv5te-nspire-eabi/release/deps/async_demo-4f8ee0faccd8ac66.async_demo.42x0nugk-cgu.4.rcgu.o" "-o" "/home/benschattinger/Documents/Projects/Calculator/example-nspire/target/armv5te-nspire-eabi/release/deps/async_demo-4f8ee0faccd8ac66" "-Wl,--gc-sections" "-nodefaultlibs" "-L" "/home/benschattinger/Documents/Projects/Calculator/example-nspire/target/armv5te-nspire-eabi/release/deps" "-L" "/home/benschattinger/Documents/Projects/Calculator/example-nspire/target/release/deps" "-L" "/home/benschattinger/Documents/Projects/Calculator/example-nspire/target/sysroot/lib/rustlib/armv5te-nspire-eabi/lib" "-Wl,--start-group" "-Wl,--end-group" "-Wl,-Bstatic" "/home/benschattinger/Documents/Projects/Calculator/example-nspire/target/sysroot/lib/rustlib/armv5te-nspire-eabi/lib/libcompiler_builtins-3330920514730093.rlib" "-Wl,-Bdynamic" "-Wl,--allow-multiple-definition"
  = note: arm-none-eabi-ld: /home/benschattinger/Documents/Projects/Calculator/example-nspire/target/sysroot/lib/rustlib/armv5te-nspire-eabi/lib/libcompiler_builtins-3330920514730093.rlib: error adding symbols: file format not recognized
          collect2: error: ld returned 1 exit status

This occurs both with cargo-xbuild and -Zbuild-std=core,alloc.

@ehuss ehuss added A-linkage Area: linker issues, dylib, cdylib, shared libraries, so Z-build-std Nightly: build-std labels May 16, 2020
@alexcrichton
Copy link
Member

Cargo recently had changes with LTO and optimizing build times by producing object files that are actually LLVM bitcode. I suspect this is a bug where LLVM bitcode is making its way to the linker when it shouldn't.

Would it be possible to minimize this to a small Cargo project perhaps?

@alexcrichton
Copy link
Member

Ok I got a chance to dig into this a bit. This is definitely -Zbuild-std-specific.

The problem here is that Cargo is compiling the crate graph with -Clinker-plugin-lto, but crates like compiler-builtins and rlibc do not participate in LTO because they're marked with #![no_builtins].

I think there's a few ways we could fix this:

  • Update rustc to ignore -Clinker-plugin-lto if the crate is marked with #![no_builtins]. I don't think it's basically ever correct for these crates to participate in LTO.
  • Update Cargo to learn about #![no_builtins] and don't pass these flags.
  • Update Cargo with specific knowledge about compiler-builtins and leave rlibc broken.

I'll see if I can whip up a patch to do the first of these.

alexcrichton added a commit to alexcrichton/rust that referenced this issue May 18, 2020
This commit updates the code generation for `#![no_builtins]` to always
produce object files instead of conditionally respecting
`-Clinker-plugin-lto` and sometimes producing bitcode. This is intended
to address rust-lang/cargo#8239.

The issue at hand here is that Cargo has tried to get "smarter" about
codegen in whole crate graph scenarios. When LTO is enabled it attempts
to avoid codegen on as many crates as possible, opting to pass
`-Clinker-plugin-lto` where it can to only generate bitcode. When this
is combined with `-Zbuild-std`, however, it means that
`compiler-builtins` only generates LLVM bitcode instead of object files.
Rustc's own LTO passes then explicitly skip `compiler-builtins` (because
it wouldn't work anyway) which means that LLVM bitcode gets sent to the
linker, which chokes most of the time.

The fix in this PR is to not actually respect `-Clinker-plugin-lto` for
`#![no_builtins]` crates. These crates, even if slurped up by the linker
rather than rustc, will not work with LTO. They define symbols which are
only referenced as part of codegen, so LTO's aggressive internalization
would trivially remove the symbols only to have the linker realize later
that the symbol is undefined. Since pure-bitcode never makes sense for
these libraries, the `-Clinker-plugin-lto` flag is silently ignored.
@alexcrichton
Copy link
Member

Ok this should be fixed with rust-lang/rust#72325

RalfJung added a commit to RalfJung/rust that referenced this issue May 22, 2020
…to, r=nnethercote

Always generated object code for `#![no_builtins]`

This commit updates the code generation for `#![no_builtins]` to always
produce object files instead of conditionally respecting
`-Clinker-plugin-lto` and sometimes producing bitcode. This is intended
to address rust-lang/cargo#8239.

The issue at hand here is that Cargo has tried to get "smarter" about
codegen in whole crate graph scenarios. When LTO is enabled it attempts
to avoid codegen on as many crates as possible, opting to pass
`-Clinker-plugin-lto` where it can to only generate bitcode. When this
is combined with `-Zbuild-std`, however, it means that
`compiler-builtins` only generates LLVM bitcode instead of object files.
Rustc's own LTO passes then explicitly skip `compiler-builtins` (because
it wouldn't work anyway) which means that LLVM bitcode gets sent to the
linker, which chokes most of the time.

The fix in this PR is to not actually respect `-Clinker-plugin-lto` for
`#![no_builtins]` crates. These crates, even if slurped up by the linker
rather than rustc, will not work with LTO. They define symbols which are
only referenced as part of codegen, so LTO's aggressive internalization
would trivially remove the symbols only to have the linker realize later
that the symbol is undefined. Since pure-bitcode never makes sense for
these libraries, the `-Clinker-plugin-lto` flag is silently ignored.
@ehuss
Copy link
Contributor

ehuss commented Jun 5, 2020

I'm going to close this as fixed, per rust-lang/rust#72325.

There's still some open questions about how compiler-builtins symbols are working -- whether or not it uses the "mem" feature, particularly for JSON spec targets, and how those symbols are exported. I still get undefined references from memcpy to various things in libcore even with the mem feature. Tracking those in rust-lang/wg-cargo-std-aware#53.

@eira-fransham
Copy link

I'm getting this issue when compiling under aarch64 Arch Linux (I don't know if it happens on other distros because they have too-old a version of ld and it causes some problems with the latest Clang version). I'm compiling C++ code and linking to it from Rust. Disabling LTO in both clang and cargo fixes the issue. Under x86_64 Arch Linux it works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-linkage Area: linker issues, dylib, cdylib, shared libraries, so C-bug Category: bug Z-build-std Nightly: build-std
Projects
None yet
Development

No branches or pull requests

5 participants