Problems with linking with system-wide installed libraries

[jauhien]

Short description

Rust linking system has one big problem now, that blocks possibility to have a given rust package to be installed and to be developed on the same system.

tl;dr: when you have a library installed system wide and you are developing package that contains this library (and uses other system wide installed libs) you'll have problems with linking because of

error: multiple dylib candidates for `hello_world` found

Example projects

Consider the minimal example, think we have those projects:

1. A library
2. A project that has executables + libraries

A library project looks like this:

jauhien@zcj linking % cat hello_lib/hello_lib.rs 
#![crate_type = "dylib"]

pub fn hello(name: &str) -> String {
    "Hello, ".to_string() + name + "!".to_string()
}

A project with binary and library looks like this:

jauhien@zcj linking % cat hello_world/hello_world.rs 
#![crate_type = "dylib"]

extern crate hello_lib;

use hello_lib::hello;

pub fn hello_name(name: &str, surname: &str) -> String {
    hello((name.to_string() + " " + surname).as_slice())
}

jauhien@zcj linking % cat hello_world/hello.rs 
#![crate_type = "bin"]

extern crate hello_world;

use hello_world::hello_name;

fn main() {
   println!("{}", hello_name("Test", "Test"))
}

Steps to reproduce the error

I use 'image' directory as root here.

1. Compile and install hello_lib library

   jauhien@zcj hello_lib % rustc hello_lib.rs 
   jauhien@zcj hello_lib % cp libhello_lib.so ../image/usr/lib 
   

2. Compile and install hello_world project that uses hello_lib

   jauhien@zcj hello_world % rustc hello_world.rs -L ../image/usr/lib
   jauhien@zcj hello_world % rustc hello.rs -L . -L ../image/usr/lib 
   jauhien@zcj hello_world % cp libhello_world.so ../image/usr/lib 
   jauhien@zcj hello_world % cp hello ../image/usr/bin 
   

   Test if it works:

   jauhien@zcj hello_world % LD_LIBRARY_PATH=../image/usr/lib ../image/usr/bin/hello
   Hello, Test Test!         
   

3. Continue developing of hello_world project and try to recompile it

   jauhien@zcj hello_world % rustc hello_world.rs -L ../image/usr/lib
   jauhien@zcj hello_world % rustc hello.rs -L . -L ../image/usr/lib 
   hello.rs:3:1: 3:26 error: multiple dylib candidates for `hello_world` found
   hello.rs:3 extern crate hello_world;
              ^~~~~~~~~~~~~~~~~~~~~~~~~
   hello.rs:3:1: 3:26 note: candidate #1: /home/jauhien/work/rust/linking/hello_world/libhello_world.so
   hello.rs:3 extern crate hello_world;
              ^~~~~~~~~~~~~~~~~~~~~~~~~
   hello.rs:3:1: 3:26 note: candidate #2: /home/jauhien/work/rust/linking/image/usr/lib/libhello_world.so
   hello.rs:3 extern crate hello_world;
              ^~~~~~~~~~~~~~~~~~~~~~~~~
   error: aborting due to previous error
   

Conclusions

So we have problem with current linking system. It is bad because

• it makes not possible to develop rust software on the machine that has already installed version of this software (as rust infrastructure will grow, nearly every project will use libraries from system)
• it leads to problems with packaging rust software (and rust compiler itself), see e.g. #13421, https://github.com/Heather/gentoo-rust/issues/19 and #13733

I think that this issue can be solved if some algorithm that chooses which dynamic library to use depending on the order of -L flags is introduced to linking code. That means, e.g. use libraries found under the earliest -L path and ignore others.

[alexcrichton]

I talked with @brson about this yesterday, and we think that we should try out the following scheme:

1. If two libraries are found in two different locations (via -L), the first mentioned on the command line (far left) is chosen silently. This mirrors what gcc/clang do.
2. If two libraries are found in the same location, an error is emitted.

The current behavior is a relic of an era since past, so it's certainly ripe for changing.

[jauhien]

Yes, this is reasonable solution. It would be nice if this would be implemented.

[Siosm]

According to me this should be marked as in need of a fix before the 1.0 release.

[bheesham]

I'm getting a similar error when trying to build with openssl = "~0.2.2". Full output: https://clbin.com/19aX8

Steps to reproduce:

1. cargo new test --bin
2. Add openssl = "~0.2.2" as a dependency.
3. cargo build

[msierks]

I am also getting this error. Is there a workaround ?

[jauhien]

@msierks, @bheesham: the workaround is installation of the Rust libraries not in the system location. You can use --libdir configure switch for this. Or you can install the whole Rust in different location (and if you are compiling it by hand, I would highly recommend install it in /usr/local as any software compiled by hand).

The first installation way is used e.g. here: https://github.com/Heather/gentoo-rust/tree/master/dev-lang/rust

[msierks]

@jauhien: worked, thanks.

[eagleflo]

rust-sdl2 recently transitioned into using pkg-config and now can't be compiled on OS X thanks to this issue, as both Rust and SDL2 (from Homebrew) are installed into /usr/local/lib by default and pkg-config passes in -L /usr/local/lib to rustc.

Here's a gist with the exact error: https://gist.github.com/eagleflo/d6d7f446a66ed37d7869.

[Keruspe]

Any new on this?

Even the rust compiler itself cannot be compiled if it's already installed system-wide...

[brson]

In Debian's packaging solution for Rust they are not planning on installing Rust libraries globally, only Rust applications, so will not hit this problem. They will be packaging Rust libraries as source to a custom system location. When their package Rust applications are built, they will redirect cargo to look in that location for the source code to their dependencies, which will then be built and statically linked to the application. But the libraries will never be installed anywhere that could cause such a conflict.

[johnzachary]

Please consider fixing this. It pops when trying to build statically linked programs that depend on libraries in /usr/local/lib. I don't think I should have to rebuild the rust compiler or install some things in non-standard locations to use a language feature.

[snf]

I've just hit this bug with rustc 1.9.0-nightly (e91f889 2016-03-03):

It looks like Rust is installing these libs twice and when I add /usr/local/lib for another locally installed lib I hit it:

error: multiple dylib candidates for `std` found [E0465]
note: candidate #1: /usr/local/lib/libstd-18402db3.so
note: candidate #2: /usr/local/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-18402db3.so

[MarkusJais]

I ran into the same problem as snf with Rust 1.7 (clean install) and trying to build cassandra-sys-rs.
(Ubuntu 15.10.)

src/lib.rs:1:1: 1:1 error: multiple dylib candidates for `std` found [E0465]
src/lib.rs:1 #![allow(non_camel_case_types)]
             ^
src/lib.rs:1:1: 1:1 note: candidate #1: /usr/local/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-6a154fe0.so
src/lib.rs:1 #![allow(non_camel_case_types)]
             ^
src/lib.rs:1:1: 1:1 note: candidate #2: /usr/local/lib/libstd-6a154fe0.so
src/lib.rs:1 #![allow(non_camel_case_types)]

[trufae]

This can be "fixed" by using -L native=/usr/local/lib instead of -L /usr/local/lib

[fluffels]

@MarkusJais did you find a fix for this?

[fluffels]

@MarkusJais using Ubuntu's native rustc / cargo packages over rustup.sh fixed this for me

[fluffels]

I get

error: multiple dylib candidates for `std` found [E0465]
note: candidate #1: /usr/local/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-d16b8f0e.so
note: candidate #2: /usr/local/lib/libstd-d16b8f0e.so

This is apparently resolved by deleting the latter candidate. Is this going to cause issues for me later? Or is it a viable workaround?

[sunnystormy]

Same issue here. Tried to cargo install bindgen so I could begin porting a project, but the installer wouldn't finish because of the dual "libstd" problem.

[steveklabnik]

Triage: with the last comment here being a year and a half old, and with a lot of people seemingly having different but similar issues, and with many having fixed their issues, I'm going to give this one a close. If anyone on this thread is still having this problem today, please open a new bug with specific details, thanks.