Problems with linking with system-wide installed libraries #16402

Open
jauhien opened this Issue Aug 10, 2014 · 18 comments

Comments

Projects
None yet
@jauhien
Contributor

jauhien commented Aug 10, 2014

Short description

Rust linking system has one big problem now, that blocks possibility to have a given rust package to be installed and to be developed on the same system.

tl;dr: when you have a library installed system wide and you are developing package that contains this library (and uses other system wide installed libs) you'll have problems with linking because of

error: multiple dylib candidates for `hello_world` found

Example projects

Consider the minimal example, think we have those projects:

  1. A library
  2. A project that has executables + libraries

A library project looks like this:

jauhien@zcj linking % cat hello_lib/hello_lib.rs 
#![crate_type = "dylib"]

pub fn hello(name: &str) -> String {
    "Hello, ".to_string() + name + "!".to_string()
}

A project with binary and library looks like this:

jauhien@zcj linking % cat hello_world/hello_world.rs 
#![crate_type = "dylib"]

extern crate hello_lib;

use hello_lib::hello;

pub fn hello_name(name: &str, surname: &str) -> String {
    hello((name.to_string() + " " + surname).as_slice())
}
jauhien@zcj linking % cat hello_world/hello.rs 
#![crate_type = "bin"]

extern crate hello_world;

use hello_world::hello_name;

fn main() {
   println!("{}", hello_name("Test", "Test"))
}

Steps to reproduce the error

I use 'image' directory as root here.

  1. Compile and install hello_lib library

    jauhien@zcj hello_lib % rustc hello_lib.rs 
    jauhien@zcj hello_lib % cp libhello_lib.so ../image/usr/lib 
    
  2. Compile and install hello_world project that uses hello_lib

    jauhien@zcj hello_world % rustc hello_world.rs -L ../image/usr/lib
    jauhien@zcj hello_world % rustc hello.rs -L . -L ../image/usr/lib 
    jauhien@zcj hello_world % cp libhello_world.so ../image/usr/lib 
    jauhien@zcj hello_world % cp hello ../image/usr/bin 
    

    Test if it works:

    jauhien@zcj hello_world % LD_LIBRARY_PATH=../image/usr/lib ../image/usr/bin/hello
    Hello, Test Test!         
    
  3. Continue developing of hello_world project and try to recompile it

    jauhien@zcj hello_world % rustc hello_world.rs -L ../image/usr/lib
    jauhien@zcj hello_world % rustc hello.rs -L . -L ../image/usr/lib 
    hello.rs:3:1: 3:26 error: multiple dylib candidates for `hello_world` found
    hello.rs:3 extern crate hello_world;
               ^~~~~~~~~~~~~~~~~~~~~~~~~
    hello.rs:3:1: 3:26 note: candidate #1: /home/jauhien/work/rust/linking/hello_world/libhello_world.so
    hello.rs:3 extern crate hello_world;
               ^~~~~~~~~~~~~~~~~~~~~~~~~
    hello.rs:3:1: 3:26 note: candidate #2: /home/jauhien/work/rust/linking/image/usr/lib/libhello_world.so
    hello.rs:3 extern crate hello_world;
               ^~~~~~~~~~~~~~~~~~~~~~~~~
    error: aborting due to previous error
    

Conclusions

So we have problem with current linking system. It is bad because

  • it makes not possible to develop rust software on the machine that has already installed version of this software (as rust infrastructure will grow, nearly every project will use libraries from system)
  • it leads to problems with packaging rust software (and rust compiler itself), see e.g. #13421, https://github.com/Heather/gentoo-rust/issues/19 and #13733

I think that this issue can be solved if some algorithm that chooses which dynamic library to use depending on the order of -L flags is introduced to linking code. That means, e.g. use libraries found under the earliest -L path and ignore others.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Aug 21, 2014

Member

I talked with @brson about this yesterday, and we think that we should try out the following scheme:

  1. If two libraries are found in two different locations (via -L), the first mentioned on the command line (far left) is chosen silently. This mirrors what gcc/clang do.
  2. If two libraries are found in the same location, an error is emitted.

The current behavior is a relic of an era since past, so it's certainly ripe for changing.

Member

alexcrichton commented Aug 21, 2014

I talked with @brson about this yesterday, and we think that we should try out the following scheme:

  1. If two libraries are found in two different locations (via -L), the first mentioned on the command line (far left) is chosen silently. This mirrors what gcc/clang do.
  2. If two libraries are found in the same location, an error is emitted.

The current behavior is a relic of an era since past, so it's certainly ripe for changing.

@jauhien

This comment has been minimized.

Show comment
Hide comment
@jauhien

jauhien Aug 21, 2014

Contributor

Yes, this is reasonable solution. It would be nice if this would be implemented.

Contributor

jauhien commented Aug 21, 2014

Yes, this is reasonable solution. It would be nice if this would be implemented.

@Siosm

This comment has been minimized.

Show comment
Hide comment
@Siosm

Siosm Nov 26, 2014

Contributor

According to me this should be marked as in need of a fix before the 1.0 release.

Contributor

Siosm commented Nov 26, 2014

According to me this should be marked as in need of a fix before the 1.0 release.

@bheesham

This comment has been minimized.

Show comment
Hide comment
@bheesham

bheesham Dec 3, 2014

Contributor

I'm getting a similar error when trying to build with openssl = "~0.2.2". Full output: https://clbin.com/19aX8

Steps to reproduce:

  1. cargo new test --bin
  2. Add openssl = "~0.2.2" as a dependency.
  3. cargo build
Contributor

bheesham commented Dec 3, 2014

I'm getting a similar error when trying to build with openssl = "~0.2.2". Full output: https://clbin.com/19aX8

Steps to reproduce:

  1. cargo new test --bin
  2. Add openssl = "~0.2.2" as a dependency.
  3. cargo build
@msierks

This comment has been minimized.

Show comment
Hide comment
@msierks

msierks Dec 4, 2014

I am also getting this error. Is there a workaround ?

msierks commented Dec 4, 2014

I am also getting this error. Is there a workaround ?

@jauhien

This comment has been minimized.

Show comment
Hide comment
@jauhien

jauhien Dec 4, 2014

Contributor

@msierks, @bheesham: the workaround is installation of the Rust libraries not in the system location. You can use --libdir configure switch for this. Or you can install the whole Rust in different location (and if you are compiling it by hand, I would highly recommend install it in /usr/local as any software compiled by hand).

The first installation way is used e.g. here: https://github.com/Heather/gentoo-rust/tree/master/dev-lang/rust

Contributor

jauhien commented Dec 4, 2014

@msierks, @bheesham: the workaround is installation of the Rust libraries not in the system location. You can use --libdir configure switch for this. Or you can install the whole Rust in different location (and if you are compiling it by hand, I would highly recommend install it in /usr/local as any software compiled by hand).

The first installation way is used e.g. here: https://github.com/Heather/gentoo-rust/tree/master/dev-lang/rust

@msierks

This comment has been minimized.

Show comment
Hide comment
@msierks

msierks Dec 4, 2014

@jauhien: worked, thanks.

msierks commented Dec 4, 2014

@jauhien: worked, thanks.

@eagleflo

This comment has been minimized.

Show comment
Hide comment
@eagleflo

eagleflo Jan 3, 2015

rust-sdl2 recently transitioned into using pkg-config and now can't be compiled on OS X thanks to this issue, as both Rust and SDL2 (from Homebrew) are installed into /usr/local/lib by default and pkg-config passes in -L /usr/local/lib to rustc.

Here's a gist with the exact error: https://gist.github.com/eagleflo/d6d7f446a66ed37d7869.

eagleflo commented Jan 3, 2015

rust-sdl2 recently transitioned into using pkg-config and now can't be compiled on OS X thanks to this issue, as both Rust and SDL2 (from Homebrew) are installed into /usr/local/lib by default and pkg-config passes in -L /usr/local/lib to rustc.

Here's a gist with the exact error: https://gist.github.com/eagleflo/d6d7f446a66ed37d7869.

@SimonSapin SimonSapin referenced this issue in michaelsproul/rust-nightly-archlinux Jan 19, 2015

Closed

Why install to /usr again? #13

@Keruspe

This comment has been minimized.

Show comment
Hide comment
@Keruspe

Keruspe Aug 7, 2015

Contributor

Any new on this?

Even the rust compiler itself cannot be compiled if it's already installed system-wide...

Contributor

Keruspe commented Aug 7, 2015

Any new on this?

Even the rust compiler itself cannot be compiled if it's already installed system-wide...

@brson

This comment has been minimized.

Show comment
Hide comment
@brson

brson Dec 28, 2015

Contributor

In Debian's packaging solution for Rust they are not planning on installing Rust libraries globally, only Rust applications, so will not hit this problem. They will be packaging Rust libraries as source to a custom system location. When their package Rust applications are built, they will redirect cargo to look in that location for the source code to their dependencies, which will then be built and statically linked to the application. But the libraries will never be installed anywhere that could cause such a conflict.

Contributor

brson commented Dec 28, 2015

In Debian's packaging solution for Rust they are not planning on installing Rust libraries globally, only Rust applications, so will not hit this problem. They will be packaging Rust libraries as source to a custom system location. When their package Rust applications are built, they will redirect cargo to look in that location for the source code to their dependencies, which will then be built and statically linked to the application. But the libraries will never be installed anywhere that could cause such a conflict.

@johnzachary

This comment has been minimized.

Show comment
Hide comment
@johnzachary

johnzachary Jan 4, 2016

Please consider fixing this. It pops when trying to build statically linked programs that depend on libraries in /usr/local/lib. I don't think I should have to rebuild the rust compiler or install some things in non-standard locations to use a language feature.

Please consider fixing this. It pops when trying to build statically linked programs that depend on libraries in /usr/local/lib. I don't think I should have to rebuild the rust compiler or install some things in non-standard locations to use a language feature.

@snf

This comment has been minimized.

Show comment
Hide comment
@snf

snf Mar 19, 2016

Contributor

I've just hit this bug with rustc 1.9.0-nightly (e91f889 2016-03-03):

It looks like Rust is installing these libs twice and when I add /usr/local/lib for another locally installed lib I hit it:

error: multiple dylib candidates for `std` found [E0465]
note: candidate #1: /usr/local/lib/libstd-18402db3.so
note: candidate #2: /usr/local/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-18402db3.so
Contributor

snf commented Mar 19, 2016

I've just hit this bug with rustc 1.9.0-nightly (e91f889 2016-03-03):

It looks like Rust is installing these libs twice and when I add /usr/local/lib for another locally installed lib I hit it:

error: multiple dylib candidates for `std` found [E0465]
note: candidate #1: /usr/local/lib/libstd-18402db3.so
note: candidate #2: /usr/local/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-18402db3.so
@MarkusJais

This comment has been minimized.

Show comment
Hide comment
@MarkusJais

MarkusJais Mar 22, 2016

I ran into the same problem as snf with Rust 1.7 (clean install) and trying to build cassandra-sys-rs.
(Ubuntu 15.10.)

src/lib.rs:1:1: 1:1 error: multiple dylib candidates for `std` found [E0465]
src/lib.rs:1 #![allow(non_camel_case_types)]
             ^
src/lib.rs:1:1: 1:1 note: candidate #1: /usr/local/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-6a154fe0.so
src/lib.rs:1 #![allow(non_camel_case_types)]
             ^
src/lib.rs:1:1: 1:1 note: candidate #2: /usr/local/lib/libstd-6a154fe0.so
src/lib.rs:1 #![allow(non_camel_case_types)]

I ran into the same problem as snf with Rust 1.7 (clean install) and trying to build cassandra-sys-rs.
(Ubuntu 15.10.)

src/lib.rs:1:1: 1:1 error: multiple dylib candidates for `std` found [E0465]
src/lib.rs:1 #![allow(non_camel_case_types)]
             ^
src/lib.rs:1:1: 1:1 note: candidate #1: /usr/local/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-6a154fe0.so
src/lib.rs:1 #![allow(non_camel_case_types)]
             ^
src/lib.rs:1:1: 1:1 note: candidate #2: /usr/local/lib/libstd-6a154fe0.so
src/lib.rs:1 #![allow(non_camel_case_types)]
@trufae

This comment has been minimized.

Show comment
Hide comment
@trufae

trufae Mar 27, 2016

This can be "fixed" by using -L native=/usr/local/lib instead of -L /usr/local/lib

trufae commented Mar 27, 2016

This can be "fixed" by using -L native=/usr/local/lib instead of -L /usr/local/lib

@fluffels

This comment has been minimized.

Show comment
Hide comment
@fluffels

fluffels Jun 19, 2016

@MarkusJais did you find a fix for this?

@MarkusJais did you find a fix for this?

@fluffels

This comment has been minimized.

Show comment
Hide comment
@fluffels

fluffels Jun 19, 2016

@MarkusJais using Ubuntu's native rustc / cargo packages over rustup.sh fixed this for me

@MarkusJais using Ubuntu's native rustc / cargo packages over rustup.sh fixed this for me

@fluffels

This comment has been minimized.

Show comment
Hide comment
@fluffels

fluffels Jun 19, 2016

I get

error: multiple dylib candidates for `std` found [E0465]
note: candidate #1: /usr/local/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-d16b8f0e.so
note: candidate #2: /usr/local/lib/libstd-d16b8f0e.so

This is apparently resolved by deleting the latter candidate. Is this going to cause issues for me later? Or is it a viable workaround?

I get

error: multiple dylib candidates for `std` found [E0465]
note: candidate #1: /usr/local/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-d16b8f0e.so
note: candidate #2: /usr/local/lib/libstd-d16b8f0e.so

This is apparently resolved by deleting the latter candidate. Is this going to cause issues for me later? Or is it a viable workaround?

@sunnystormy

This comment has been minimized.

Show comment
Hide comment
@sunnystormy

sunnystormy Sep 30, 2016

Same issue here. Tried to cargo install bindgen so I could begin porting a project, but the installer wouldn't finish because of the dual "libstd" problem.

sunnystormy commented Sep 30, 2016

Same issue here. Tried to cargo install bindgen so I could begin porting a project, but the installer wouldn't finish because of the dual "libstd" problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment