building cargo atop multirust fails, dyn link problems (Mac OS X)

[pnkfelix]

building cargo atop multirust on Mac OS X fails during build-script-build invocations

error: aborting due to 6 previous errors
failed to run custom build command for `time v0.1.23`
Process didn't exit successfully: `/Users/pnkfelix/Dev/Mozilla/cargo-dbgopt.git/target/debug/build/time-6a09e66e4b5c9d32/build-script-build` (signal: 5)
--- stderr
dyld: Library not loaded: x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib/libstd-4e7c5e5c.dylib
  Referenced from: /Users/pnkfelix/Dev/Mozilla/cargo-dbgopt.git/target/debug/build/time-6a09e66e4b5c9d32/build-script-build
  Reason: image not found

I do not know whether this is a cargo bug or a multirust one, to be honest, but it seemed troubling that the attempts to resolve libstd were not going through the DYLD_LIBRARY_PATH and instead seemed to be going through ... what I think is an embedded rpath ?

(It is also weird that it seems like is using this path x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib/libstd-4e7c5e5c.dylib which looks like it must be coming from a local rust build, even though I am attempting to use multirust solely with the beta and nightly channels ... perhaps that is a clue to something more deeply wrong in my setup...)

[brson]

Baffling to me as well on first glance. I've got a backlog of multirust bugs to fix and I'll try to get to this in the next couple of weeks. Let me know if it's blocking you or if you see it happen elsewhere.

multirust has had trouble building cargo itself before.

[brson]

I did this on OS X to build Cargo with multirust successfully:

multirust default nightly-2015-02-20 # Cargo builds with a very old Rust.
./configure --local-rust-root=~/.multirust/toolchains/nightly-2015-02-20
make

That works because Cargo's Makefiles set up DYLD_LIBRARY_PATH.

If I try to execute the in-tree cargo directly while putting multirust's rustc on the path I get an error like yours

"/Users/administrator/.multirust/toolchains/nightly-2015-02-20//bin/rustc" -V
rustc 1.0.0-nightly (522d09dfe 2015-02-19) (built 2015-02-19)
"target/snapshot/bin/cargo" --version
cargo 0.0.1-pre-nightly (bb28e71 2015-01-22 06:06:34 +0000)
target/snapshot/bin/cargo build --target x86_64-apple-darwin   
   Compiling cargo v0.1.0 (file:///Users/administrator/scratch/cargo)
rust-mac6:cargo administrator$ target/snapshot/bin/cargo build
Could not execute process `rustc -v verbose` (status=never executed)

Caused by:
  no such file or directory
rust-mac6:cargo administrator$ PATH=$PATH:~/.multirust/toolchains/nightly-2015-02-20/bin/ target/snapshot/bin/cargo build
Process didn't exit successfully: `rustc -v verbose` (status=5)
--- stderr
dyld: Library not loaded: x86_64-apple-darwin/stage1/lib/rustlib/x86_64-apple-darwin/lib/librustc_driver-4e7c5e5c.dylib
  Referenced from: /Users/administrator/.multirust/toolchains/nightly-2015-02-20/bin//rustc
  Reason: image not found

That's because it's using Cargo's own snapshot of cargo to do the build. It's not part of multirust and doesn't configure the environment to find the multirust-installed toolchain.

Setting both PATH and DYLD_LIBRARY_PATH before invoking the snapshot cargo works.

PATH=$PATH:~/.multirust/toolchains/nightly-2015-02-20/bin/ DYLD_LIBRARY_PATH=~/.multirust/toolchains/nightly-2015-02-20/lib/ target/snapshot/bin/cargo build

How are you invoking the cargo build?

[pnkfelix]

Hmm I think my wrapper script only mucks with DYLD_LIBRARY_PATH, not PATH. but I'll have to review

[pnkfelix]

@brson BTW, how did you find out the specific nightly to use to build cargo? I've often been flummoxed by my inability to figure out which version of rustc the head of cargo was last built upon.

[brson]

@pnkfelix Cargo has a file src/rustversion.txt which corresponds to the nightly archive date.

[killercup]

I have a similar problem with nrc/rustfmt. Executing the binary from the target/release/ directory requires me to explicitly set DYLD_LIBRARY_PATH.

[pnkfelix]

I'm going to close this; @brson's feedback gave me enough info to get cargo building, and I'm not sure there's something here that is actually actionable, at least from the point-of-view of multirust itself.

[brson]

Well, I reopened this because @killercup's description of his bug made me suspicious that rpath is not working as expected.

[killercup]

@brson, come to think of it, I'm not sure my problem is actually about multirust.

Using cargo run works fine. I guess the multirust wrapper around cargo sets the correct DYLD_LIBRARY_PATH. By default, this env variable is empty, by the way.

When executing the binary from target/release/, multirust should not be involved at all, right? This is when I get the dyld: Library not loaded error.

The Rust libraries in multirust's toochains are not in a default path like /usr/local/lib/, so dyld has no idea where to find them, except if their (absolute) path was included in the binary. This should not be a problem when using a OS package manager or rustup.sh.

I don't know how to work around this except for telling people to set their DYLD_LIBRARY_PATH themselves (e.g. using a notice after multirust's install). (Or telling binary authors to not dynamically link to Rust's libraries if that is an option.)

[brson]

@killercup Yes, when you run the bin produced by cargo multirust isn't invoked, but multirust passes -C rpath to all calls to rustc, which is supposed to configure the dynamic linker so that things just work when the binary is executed.

It's possible that Cargo is moving binaries around in a way that breaks -C rpath.

[brson]

BTW, if you want to configure the environment so that the dynamic linker works for a particular toolchain, multirust run beta bash should do it, though I haven't tested a scenario like this.

[paulp]

which is supposed to configure the dynamic linker so that things just work when the binary is executed.

I don't think it's working at all. Are there tests which exercise it?

I don't see any of @executable_path, @loader_path, or @rpath in the shared library paths. Is there a facility for things to just work when there's no virtual path placeholder?

% otool -L ./target/debug/rusti
./target/debug/rusti:
  x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib/librustc_driver-11582ce5.dylib (compatibility version 0.0.0, current version 0.0.0)
  x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib/librustc_privacy-11582ce5.dylib (compatibility version 0.0.0, current version 0.0.0)
  x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib/librustc_resolve-11582ce5.dylib (compatibility version 0.0.0, current version 0.0.0)
  ...
  /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)
  /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0)

For contrast, here's what I expect to see:

% otool -L Alfred\ 2.app/Contents/MacOS/Alfred\ 2
  @executable_path/../Frameworks/Growl.framework/Versions/A/Growl 
  ...
  @rpath/Alfred Framework.framework/Versions/A/Alfred Framework
  ...

BTW, if you want to configure the environment so that the dynamic linker works for a particular toolchain, multirust run beta bash should do it, though I haven't tested a scenario like this.

That does work, by setting DYLD_LIBRARY_PATH.

Edit: I believe rust-lang/rust#17219 is about the same issue.

[brson]

Yes, rust-lang/rust#17219 seems to be the most likely culprit.

[alexcrichton]

Note that Cargo no longer builds build scripts dynamically so this issue may just be moot now.

[seanjensengrey]

I am still seeing the incorrect lib paths in binaries

multirust: nightly revision:

rustc 1.5.0-nightly (eafe106ef 2015-10-15)
cargo 0.6.0-nightly (7b630d4 2015-10-12)

seanj ~/w/rust/rustbook ▶ multirust override nightly
seanj ~/w/rust/rustbook ▶ cargo clean
seanj ~/w/rust/rustbook ▶ cargo build --release
   Compiling rustbook v0.4.0 (file:///Volumes/lake0/seanj/w/rust/rustbook)
seanj ~/w/rust/rustbook ▶ otool -L target/release/rustbook 
target/release/rustbook:
    x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib/librustdoc-10cbabc2.dylib (compatibility version 0.0.0, current version 0.0.0)
    x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib/libtest-10cbabc2.dylib (compatibility version 0.0.0, current version 0.0.0)
    x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib/librustc_driver-10cbabc2.dylib (compatibility version 0.0.0, current version 0.0.0)
    x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib/librustc_lint-10cbabc2.dylib (compatibility version 0.0.0, current version 0.0.0)
    x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib/librustc_trans-10cbabc2.dylib (compatibility version 0.0.0, current version 0.0.0)
    x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib/librustc_privacy-10cbabc2.dylib (compatibility version 0.0.0, current version 0.0.0)
    x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib/librustc_mir-10cbabc2.dylib (compatibility version 0.0.0, current version 0.0.0)
      ....

multirust run nightly bash sets the proper DYLD_LIBRARY_PATH

declare -x DYLD_LIBRARY_PATH="/Users/seanj/.multirust/toolchains/nightly/lib:"

[brson]

@seanjensengrey Perhaps this issue.

[seanjensengrey]

Still reading the issue you linked, but I believe part of the problem I am seeing is rustbooks using compiler infrastructure and no static version being available for linking. A simple binary does not show the above behavior.

[kindlychung]

Similar problem here: murarth/rusti#60