Cannot link to `--crate-type dylib`

[alexcrichton]

cd $(mktemp -d)
cat <<EOF > foo.rs
pub fn foo() {}
EOF
cat <<EOF > bar.rs
extern crate foo;

fn main() {
    foo::foo();
}
EOF
rustc foo.rs --crate-type dylib
rustc bar.rs -L .

error: cannot satisfy dependencies so `std` only shows up once
help: having upstream crates all available in one format will likely make this go away
error: cannot satisfy dependencies so `core` only shows up once
help: having upstream crates all available in one format will likely make this go away
error: cannot satisfy dependencies so `collections` only shows up once
help: having upstream crates all available in one format will likely make this go away
error: cannot satisfy dependencies so `rustc_unicode` only shows up once
help: having upstream crates all available in one format will likely make this go away
error: cannot satisfy dependencies so `alloc` only shows up once
help: having upstream crates all available in one format will likely make this go away
error: cannot satisfy dependencies so `rand` only shows up once
help: having upstream crates all available in one format will likely make this go away
error: cannot satisfy dependencies so `libc` only shows up once
help: having upstream crates all available in one format will likely make this go away
error: cannot satisfy dependencies so `unwind` only shows up once
help: having upstream crates all available in one format will likely make this go away
error: cannot satisfy dependencies so `panic_unwind` only shows up once
help: having upstream crates all available in one format will likely make this go away
error: cannot link together two allocators: alloc_jemalloc and alloc_system
error: aborting due to 10 previous errors

I believe that there's no reason this executable couldn't be created, it should just use all dependencies through the dylib that was created in the previous step.

[MarkSwanson]

The workaround for us was to define a process where all git repos have zero tests, and all tests are moved into another git repo.

It's critical for us that we can build a Rust DSO.
It would be nice if the test cases for a DSO could be included with the git repo that contains the DSO code.

[alexcrichton]

@MarkSwanson note that the crate type "cdylib" is most appropriate for creating a Rust DSO. If you're trying to use a dylib linking to other Rust dylibs it'll be a nonstop world of pain right now unfortunately, but cdylib should be quite smooth.

[jsgf]

I believe that there's no reason this executable couldn't be created, it should just use all dependencies through the dylib that was created in the previous step.

If this were a diamond dependency, where D depends on B and C which in turn both depend on A:

  A
 / \
B   C
 \ /
  D

would you also expect this to work?

Currently if you build B and C as dynamic, then they will each link their own static copy of A. When you try to link both into A, it will fail with error: cannot satisfy dependencies so 'A' only shows up once. In principle, if A is stateless then it doesn't matter which version gets used, or even a mix and match. But if A has state (eg lazy_static) then there's a semantic question of whether B and C should each get separate state or share the same state (and if so, how to make sure they do).

The simplest thing is if A is dynamically linked to both B and C, so a single instance is shared between B and C, and there's no need to disambiguate them.

(Edit: swapped A and D to match the answer below)

[alexcrichton]

@jsgf it depends on how all the linkages work out, we've got a test for this in tree for the various versions though. The gist is:

A	B	C	D	links?
rlib	rlib	rlib	rlib	✓
dylib	rlib	rlib	rlib	✓
rlib	dylib	rlib	rlib	✓
dylib	dylib	rlib	rlib	✓
rlib	rlib	dylib	rlib	✓
dylib	rlib	dylib	rlib	✓
rlib	dylib	dylib	rlib
dylib	dylib	dylib	rlib	✓
rlib	rlib	rlib	dylib	✓
dylib	rlib	rlib	dylib	✓
rlib	dylib	rlib	dylib	✓
dylib	dylib	rlib	dylib	✓
rlib	rlib	dylib	dylib	✓
dylib	rlib	dylib	dylib	✓
rlib	dylib	dylib	dylib
dylib	dylib	dylib	dylib	✓

The only failure mode is when B/C are both dylibs, statically linking A, so D can't be guaranteed one copy of A. In other cases A is only included once as it's statically linked into only one dylib.

The compiler isn't very smart about this today, though. It has a very simple heuristic where if it doesn't work on the first try it emits an error.

[zicklag]

Compiling it like this does work:

> rustc foo.rs --crate-type dylib -C prefer-dynamic
> rustc bar.rs -L .

But then you have to dynamically link to libstd because it won't get statically linked into libfoo.

[clouds56]

Is there any plan for the fix?
I'm creating a large application and now the linkage took more than 2 minutes.

Should I make every module cdylib? That would change all the interface IIRC.