Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bootstrap fails due to system LLVM libffi link error #34486

Open
MagaTailor opened this issue Jun 26, 2016 · 20 comments

Comments

@MagaTailor
Copy link

commented Jun 26, 2016

Trying to bootstrap rust on aarch64 Linux using --llvm-root fails:

error: linking with `cc` failed: exit code: 1
note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/bin/rustc.0.o" "-o" "aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/bin/rustc" "-Wl,--gc-sections" "-pie" "-Wl,-O1" "-nodefaultlibs" "-L" "/usr/lib/llvm-3.8/lib" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "-Wl,-Bdynamic" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "rustc_driver-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "rustc_passes-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "rustc_borrowck-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "rustc_lint-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "rustc_resolve-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "rustc_typeck-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "rustc_plugin-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "rustc_privacy-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "rustc_trans-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "rustc_platform_intrinsics-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "rustc_incremental-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "syntax_ext-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "rustc_mir-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "rustc_const_eval-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "rustc_metadata-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "rustc_save_analysis-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "rustc-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "rustc_const_math-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "getopts-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "flate-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "rustc_back-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "rbml-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "rustc_data_structures-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "syntax-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "serialize-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "log-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "term-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "fmt_macros-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "arena-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "rustc_llvm-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "graphviz-fe3cdf61" "-L" "../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-l" "std-fe3cdf61" "-l" "rt" "-l" "dl" "-l" "tinfo" "-l" "pthread" "-l" "z" "-l" "m" "-l" "stdc++" "-l" "dl" "-l" "pthread" "-l" "gcc_s" "-l" "c" "-l" "m" "-l" "rt" "-l" "util" "-Wl,-rpath,$ORIGIN/../lib" "-Wl,-rpath,/usr/local/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-Wl,--enable-new-dtags" "-s" "-l" "compiler-rt"

note: ../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib/librustc_llvm-fe3cdf61.so: undefined reference to `ffi_type_float'
../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib/librustc_llvm-fe3cdf61.so: undefined reference to `ffi_type_sint64'
../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib/librustc_llvm-fe3cdf61.so: undefined reference to `ffi_type_pointer'
../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib/librustc_llvm-fe3cdf61.so: undefined reference to `ffi_type_sint32'
../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib/librustc_llvm-fe3cdf61.so: undefined reference to `ffi_type_void'
../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib/librustc_llvm-fe3cdf61.so: undefined reference to `ffi_type_sint8'
../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib/librustc_llvm-fe3cdf61.so: undefined reference to `ffi_call'
../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib/librustc_llvm-fe3cdf61.so: undefined reference to `ffi_type_sint16'
../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib/librustc_llvm-fe3cdf61.so: undefined reference to `ffi_type_double'
../aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/lib/librustc_llvm-fe3cdf61.so: undefined reference to `ffi_prep_cif'
collect2: error: ld returned 1 exit status

error: aborting due to previous error
../mk/target.mk:220: recipe for target 'aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/bin/rustc' failed
make: *** [aarch64-unknown-linux-gnu/stage0/lib/rustlib/aarch64-unknown-linux-gnu/bin/rustc] Error 101

For some reason the correct llvm configuration is not being detected and -lffi has to be passed manually for the bootstrap to succeed.

@sanxiyn sanxiyn added the A-LLVM label Jun 27, 2016

@brson brson added the A-linkage label Jun 27, 2016

@brson

This comment has been minimized.

Copy link
Contributor

commented Jun 27, 2016

In-tree, we configure LLVM without -lffi so maybe there's a discrepancy when we encounter an LLVM with it compiled in.

@MagaTailor

This comment has been minimized.

Copy link
Author

commented Jun 27, 2016

Yup, the LLVM I was bootsrapping against was clearly built with LLVM_ENABLE_FFI.

@jethrogb

This comment has been minimized.

Copy link
Contributor

commented Jul 10, 2016

Running into the same isse. Shouldn't llvm-config output this requirement?

Btw for anyone reading this: a quick fix is to add:

#[link(name = "ffi")] extern {}

to src/librustc_llvm/lib.rs.

@japaric

This comment has been minimized.

Copy link
Member

commented Aug 17, 2016

I'm also experiencing this on x86_64 Ubuntu 16.04 using the llvm-dev package and --llvm-root.

@MagaTailor

This comment has been minimized.

Copy link
Author

commented Aug 19, 2016

The title should be edited then, and btw, the quickest non-permanent solution looks like this:
RUSTFLAGS='-C link-args=-lffi'

@japaric

This comment has been minimized.

Copy link
Member

commented Aug 27, 2016

It seems this is a duplicate of #29694

@japaric

This comment has been minimized.

Copy link
Member

commented Sep 15, 2016

Recently, I have been annoyed by this a lot. I've been boostraping rustc against system llvm a lot in different Docker containers and I have to remember to apply @jethrogb patch/fix to every rust branch/worktree I work on and then I have to be careful to not commit this fix to the branch I'm working on. 😞

Could we simply commit @jethrogb's fix into master? I tested bootstrapping rust against our llvm fork with that fix merged into master (5a5736d) and at least, on Linux, it bootstraped without problems.

P.S. @petevine's RUSTFLAGS solution didn't work for me.

@MagaTailor

This comment has been minimized.

Copy link
Author

commented Sep 15, 2016

@japaric That's all there's to it, and it definitely works for the Makefiles bootstrap.

@alexcrichton

This comment has been minimized.

Copy link
Member

commented Sep 16, 2016

This is of course not intentional, and fixes are always welcome! The trick here is detecting whether we're building against an LLVM that needs libffi. For example the in-tree LLVM doesn't and we shouldn't link to it as a result.

@cuviper

This comment has been minimized.

Copy link
Member

commented Sep 16, 2016

FWIW, Fedora's LLVM is built with FFI, but this is usually fine because libLLVM.so links to libffi.so itself. However, with llvm-static.rpm installed, llvm-config starts giving static link options (but not -lffi), and ldd -r librustc_llvm-*.so indeed shows a few undefined symbol: ffi_....

@cuviper

This comment has been minimized.

Copy link
Member

commented Sep 16, 2016

IOW I blame llvm-config, not that blame solves anything.
(I also don't like that it doesn't let me choose shared linkage even when static libraries are present...)

@MagaTailor MagaTailor changed the title System LLVM link error during AArch64 bootstrap Bootstrap fails due to system LLVM libffi link error Sep 24, 2016

@userwithuid

This comment has been minimized.

Copy link

commented Oct 30, 2016

@cuviper Regarding 'choose shared over static llvm':
If I understand correctly, since 3.9 llvm-config will prefer the combo-dynlib by default over any static libs present. And even if you don't have the combo, but say shared and static component libs installed, there are now options to specifically choose --link-static or --link-shared.

Relevant llvm commit explains it: llvm-mirror/llvm@93083d4

Doesn't fix this bug here, external llvm + ffi enabled + intentionally linking llvm static libs still triggers it, just thought it would be nice to know.

(BTW, the ffi-issue seems to be sort of a known shortcoming of llvm-config --system-libs. It only statically lists the LLVMSupport sysdeps, but ffi is the (currently only?) sysdep from another component - LLVMExecutionEngine)

@cuviper

This comment has been minimized.

Copy link
Member

commented Oct 30, 2016

@userwithuid Thanks, I wasn't aware of that! I'll try that out, and maybe we should expose those explicit link options somehow too.

@ishitatsuyuki

This comment has been minimized.

Copy link
Contributor

commented Jan 29, 2017

Please tell me the current situation; rustbuild is enabled by default, the old workarounds may not work.

@MagaTailor

This comment has been minimized.

Copy link
Author

commented Jan 30, 2017

Yeah, last time I tried it didn't even build due to some completely unfamiliar link errors. (edit: I meant exactly this)

If it's still a problem in the new build system, please open a more specific issue.

@MagaTailor MagaTailor closed this Jan 30, 2017

@ishitatsuyuki

This comment has been minimized.

Copy link
Contributor

commented Jan 30, 2017

I'm quite sure that this is still a problem; please don't blindly close it.

@jethrogb

This comment has been minimized.

Copy link
Contributor

commented Jan 30, 2017

Yes this should remain open. @MagaTailor I don't understand what you mean, this issue is quite specific.

@hussamT

This comment has been minimized.

Copy link

commented Feb 16, 2017

I went ahead and filed a new bug 39880.

@hussamT

This comment has been minimized.

Copy link

commented Jan 9, 2019

@csmoe Is this an issue again in post 1.31.1 releases? I have version 1.31.1 installed and did not see this issue when compiling rust against system llvm 7.0.1.

@cuviper

This comment has been minimized.

Copy link
Member

commented Jan 9, 2019

@hussamT IIRC it was only an issue with linking external LLVM statically, and even then only if that build is configured with -DLLVM_ENABLE_FFI:BOOL=ON. At least on Fedora rawhide, I still get:

$ rpm -q llvm
llvm-7.0.1-1.fc30.x86_64
$ llvm-config --link-static --system-libs
-lz -lrt -ldl -ltinfo -lpthread -lm

... missing -lffi, even though some of the static libs do use it. But this is really an LLVM problem, as noted in #34486 (comment), so I'm not sure we need this Rust issue open except as a "known issue, see elsewhere" tracker.

Our official Fedora rust builds are linked dynamically (--enable-llvm-link-shared), and this is fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.