Linking dylib fails in some cases, "undefined reference to `rust_begin_unwind"

[roeyskatt]

Here's a minimal code example that reproduces the error:

#![crate_name = "test"]
#![crate_type = "dylib"]

use std::ops::Div;

pub struct Test
{
    pub test:i32,
}

impl Div<Test,Test> for Test { fn div( &self, v:&Test ) -> Test { Test{ test:self.test / v.test } } }

And part of the error:

note: D:\#dev\#compilers\Rust\bin\rustlib\i686-pc-windows-gnu\lib\libcore-4e7c5e5c.rlib(core-4e7c5e5c.o):(.text+0x6b9b): undefined reference to `rust_begin_unwind'

Version:

rustc 0.13.0-nightly (45cbdec41 2014-11-07 00:02:18 +0000)
binary: rustc
commit-hash: 45cbdec4174778bf915f17561ef971c068a7fcbc
commit-date: 2014-11-07 00:02:18 +0000
host: i686-pc-windows-gnu
release: 0.13.0-nightly

This also happens on the x86_64 nightly

[vadimcn]

Duplicating all rlibs on the command line makes the error go away. Which means this is a link ordering issue between Rust rlibs.
We'll probably need to do topological sorting of the libs before passing them to the linker. :(

[alexcrichton]

I believe that this is because the program here isn't actually using any symbols from the standard library. We do actually do a topological sort of the libs on the command line. The linker command line looks like:

gcc ... program.o -lstd ... -lcore

However, this program doesn't actually use any symbols from the standard library's rlib, so the linker actually discards -lstd after it looks at it, moving on to the next library. When it reaches -lcore (which has a reverse dependency back onto libstd), we've stripped libstd, so rust_begin_unwind is lost, and it becomes and undefined symbol.

The best way to solve this... I'm not entirely sure! I've thought in the past that each object generated by rustc needs a symbol which can be referenced by objects to guarantee the linker doesn't strip any of them, which would definitely help here but is slightly heavy-handed...

[vadimcn]

@alexcrichton, if we topologically sort them, shouldn't -lstd have come after -lcore? I thought we simply piled all upstream libraries onto linker command line. Can you point me to the code that does topo-sorting?

[alexcrichton]

My current understanding of linkers leads me to believe that we need a topological sorting with the outermost dependencies at the far left and their own dependencies to the right. For example, if I have a crate c1 that links to c2 which links to c3, we have two options:

gcc c1.o -lc2 -lc3
// or
gcc c1.o -lc3 -lc2

[alexcrichton]

Oops, posted too soon!

Anyway, let's assume that c1 calls c2 functions, but does not call any c3 functions. We do know, however, that c2 calls c3 functions. This means that in the first example, there are outstanding undefined references when -lc2 is encountered, and it resolves some of them, so we keep the lib. This then keeps -lc3 as well.

In the second case, when the linker looks at -lc3 the library doesn't actually satisfy any outstanding undefined references, so the linker forgets about it, includes -lc2 and then errors with undefined references to -lc3 which it already forgot about.

The current topo-sort is here:

rust/src/librustc/metadata/cstore.rs

Lines 156 to 184 in 351f7af

	pub fn get_used_crates(&self, prefer: LinkagePreference)
	-> Vec<(ast::CrateNum, Option<Path>)> {
	let mut ordering = Vec::new();
	fn visit(cstore: &CStore, cnum: ast::CrateNum,
	ordering: &mut Vec<ast::CrateNum>) {
	if ordering.as_slice().contains(&cnum) { return }
	let meta = cstore.get_crate_data(cnum);
	for (_, &dep) in meta.cnum_map.iter() {
	visit(cstore, dep, ordering);
	}
	ordering.push(cnum);
	};
	for (&num, _) in self.metas.borrow().iter() {
	visit(self, num, &mut ordering);
	}
	ordering.as_mut_slice().reverse();
	let ordering = ordering.as_slice();
	let mut libs = self.used_crate_sources.borrow()
	.iter()
	.map(|src| (src.cnum, match prefer {
	RequireDynamic => src.dylib.clone(),
	RequireStatic => src.rlib.clone(),
	}))
	.collect::<Vec<(ast::CrateNum, Option<Path>)>>();
	libs.sort_by(|&(a, _), &(b, _)| {
	ordering.position_elem(&a).cmp(&ordering.position_elem(&b))
	});
	libs
	}

[vadimcn]

I think linker works ever simpler than that: it goes through the libraries on the command line left-to-right, and at each step tries to resolve currently outstanding symbols. It never examines libraries to the left of the current one.
So even if it had already used c2 to resolve some symbols, if there are new dependencies on c2 later down the command line, it wouldn't be able to resolve them.

[vadimcn]

@alexcrichton, I wonder if we have circular dependencies in this case. Does anything in std depend on core?

[alexcrichton]

You can't actually encode a circular dependency in Rust itself, but std/core do have a circular relationship where std depends heavily on core for all its symbols and core depends on the "weak lang items" coming from std (aka rust_begin_unwind). That's the source of the problem here sadly :(

[vadimcn]

If it's a one-off, I suppose it doesn't make sense to build the logic for dealing with circular dependencies into get_used_crates(). Could we simply add #[link_args = "-Wl,-u_rust_begin_unwind"] into libcore to tell the linker that we are going to need this symbol?

[alexcrichton]

Yeah I'm ok with adding a linker flag for specifically symbol for now.

[vadimcn]

No luck, #[link_args] doesn't get encoded into crate metadata :(
Should it have been encoded?

[vadimcn]

@roeyskatt, just make sure that you call panic!() at least once in your code and everything will be fine. :-P

[roeyskatt]

@vadimcn hehe, thanks :)

[ubsan]

I have the same issue when not building with Cargo, without rlibs, on handmade_no_std.

Wait, nevermind, I'm just being dumb.

[domgetter]

Workaround is to add println!(""); in a spare function you never use. This somehow forces the compiler to link things correctly, as far as I can tell.

    #[no_mangle]
    pub extern "C" fn mandel(x: f32, y: f32, dwell: i32) -> i32 {
      //your code here
    }
    #[allow(dead_code)]
    fn spare() { println!(""); } //adding this (doesn't have to be named "spare") makes the compilation work.
    // you don't even need to add this function signature where you're using these functions.

[jethrogb]

I'm having the same issue on Linux with cargo with a setup like the following:

Library crate corepanic:

#![no_std]

pub fn do_panic() -> ! {
    panic!("test")
}

Bin crate:

#![no_main]

extern crate corepanic;

#[no_mangle]
pub extern fn main(_argc: i32, _argv: *const *const u8) -> i32 {
    corepanic::do_panic()
}

@domgetter's workaround works in this case too

[Michael-F-Bryan]

I'm currently playing around with calling Rust code from a bunch of different languages and was having the same problem (Ubuntu Linux).

The workaround @domgetter mentioned of using something from the std library to make sure it's linked in works. I had to pub extern "C" the spare() function though to stop it from being optimised away by the compiler.

Without the spare() function:

$ rustc --version
rustc 1.14.0-nightly (16eeeac78 2016-10-18)
$ rustc prime.rs -O --crate-type cdylib
$ nm libprime.rs
00000000000183b0 T is_prime

And with the spare() function you can see everything is linked in fine:

$ nm libprime.rs
000000000004c150 T is_prime
00000000000872c0 T __rust_allocate
...
00000000000811f0 T __rust_start_panic
000000000004c1d0 T spare

Are there any special flags you can pass into rustc/cargo in order to make sure it links in the necessary things from std?

[shepmaster]

This indeed is no longer a Windows-specific issue. I believe that the introduction of cdylibs with more minimal function exporting triggers this on Linux. I still have not seen it on macOS.

@jethrogb and @Michael-F-Bryan, could you help me confirm my suspicion? Were you using dylib or cdylib?

@alexcrichton, would you mind updating the labels to reflect this change?

[alexcrichton]

Certainly!

[philippkeller]

@shepmaster: yes I'm seeing that on Linux as well now:

➜  apue git:(master) ✗ cargo build
   Compiling apue v0.0.2 (file:///home/osboxes/oss/apue)
error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/home/osboxes/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/home/osboxes/oss/apue/target/linux/debug/deps/f04_echo-1d4b104c06894c44.0.o" "-o" "/home/osboxes/oss/apue/target/linux/debug/deps/f04_echo-1d4b104c06894c44" "-Wl,--gc-sections" "-pie" "-nodefaultlibs" "-L" "/home/osboxes/oss/apue/target/linux/debug/deps" "-L" "/home/osboxes/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "-Wl,-Bdynamic" "/home/osboxes/oss/apue/target/linux/debug/deps/liblibc-73c6422e7b19c2f3.rlib" "/home/osboxes/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-b4054fae3db32020.rlib" "/home/osboxes/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-d2ecc8049920bea8.rlib" "/home/osboxes/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-5837d7d3490e00c5.rlib" "/home/osboxes/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librand-1c6ed188684e7d33.rlib" "/home/osboxes/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcollections-63f7707126c5a809.rlib" "/home/osboxes/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-0720511b45a7223a.rlib" "/home/osboxes/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_unicode-a9711770523833d4.rlib" "/home/osboxes/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc_jemalloc-477554c8f244cbba.rlib" "/home/osboxes/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-ab203041f1ec5313.rlib" "/home/osboxes/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-93f19628b61beb76.rlib" "/home/osboxes/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-35d2bc471c7ce467.rlib" "-l" "util" "-l" "dl" "-l" "rt" "-l" "pthread" "-l" "gcc_s" "-l" "pthread" "-l" "c" "-l" "m" "-l" "rt" "-l" "util"
  = note: /home/osboxes/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-93f19628b61beb76.rlib(core-93f19628b61beb76.0.o): In function `core::panicking::panic_fmt':
/buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libcore/panicking.rs:69: undefined reference to `rust_begin_unwind'
collect2: error: ld returned 1 exit status

Linux version: Linux osboxes 4.8.0-32-generic #34-Ubuntu SMP Tue Dec 13 14:30:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

I started with the second example in of no stdlib but needed to remove rust_begin_panic and rust_eh_personality otherwise I got error[E0152]: duplicate lang item found: panic_fmt. So it seems additionally to the problem above the docs seem out of date.

[binarycrusader]

Confirmed; I hit this today when building a minimal cdylib library intended for calling from C. The spare() method workaround works for now. In my case, however, I'm not using #[no_std] or #[no_main].