regression: Xargo v0.3.9 can't compile compiler-builtins

[japaric]

STR

$ cargo new --lib foo && cd $_

$ echo '#![no_std]' > src/lib.rs

$ cat >Xargo.toml <<EOF
[dependencies.core]
stage = 0

[dependencies.compiler_builtins]
stage = 1
EOF

$ xargo build
   Compiling core v0.0.0 (file://$SYSROOT/lib/rustlib/src/rust/src/libcore)
    Finished release [optimized] target(s) in 29.77 secs
error: no matching version `= 0.0.0` found for package `compiler_builtins` (required by `sysroot`)
location searched: file://$SYSROOT/lib/rustlib/src/rust/src/libcompiler_builtins
versions found: 0.1.0
consider running `cargo update` to update a path dependency's locked version
error: `"cargo" "build" "--release" "--manifest-path" "/tmp/xargo.XRHsTJQdwa4b/Cargo.toml" "--target" "x86_64-unknown-linux-gnu" "-p" "compiler_builtins"` failed with exit code: Some(101)
note: run with `RUST_BACKTRACE=1` for a backtrace 

Meta

$ rustc -V
rustc 1.22.0-nightly (368122087 2017-09-06)

$ xargo -V
xargo 0.3.9
cargo 0.23.0-nightly (34c0674a2 2017-09-01)

Diagnosis

As the error message says the libcompiler_builtins crate has version 0.1.0 in its Cargo.toml but this doesn't match the version in Cargo.lock: 0.0.0.

Why does this happen? When bootstrapping the compiler the Cargo.toml in src/libcompiler_builtins is not used; instead the Cargo.toml in src/rustc/compiler_builtins_shim is used. Because of this the version that appears in Cargo.lock is 0.0.0 -- this matches the version in src/rustc/compiler_builtins_shim/Cargo.toml. However, when Xargo compiles the Rust source it assumes that the source of dependencies for which no path or git information is specified in Xargo.toml will be located at $XARGO_RUST_SRC/lib$name/Cargo.toml where $name is the name of the dependency. Thus Xargo will use the version in src/libcompiler_builtins/Cargo.toml: 0.1.0. Hence the unmet version requirement.

What should we do?

Clearly the Xargo assumption of where the source for the compiler_builtins crate is located is not always right. We can ...

(a) continue using this assumption and patch rust-lang-nursery/compiler-builtins so that the version in its Cargo.toml matches the version in rust-lang/rust. Then we update the submodule in rust-lang/rust. Problem fixed (until someone changes either version ... as there's nothing forcing them to match)

(b) add special syntax to Xargo.toml to let the user specify the exact path of dependencies that reside in the rust-src component. Maybe: $XARGO_RUST_SRC/libcore, $XARGO_RUST_SRC/rustc/libcompiler_builtins, etc. To avoid breaking current users we use $XARGO_RUST_SRC/lib$name as the default if both path and git information is missing in Xargo.toml.

(c) do something else?

Both (a) and (b) seem rather prone to breakage. Maybe (b) more than (a) since the layout of the rust-src crates is by no means stable.

Thoughts?

cc @RalfJung

[RalfJung]

I think the version numbers should be the same really, so I feel like (a) should be done nevertheless.

But having (b) may be useful anyway? Not sure. The layout of rust-src not being stable is not exactly an argument for "it will always be $XARGO_RUST_SRC/lib$name/Cargo.toml"...

until someone changes either version ... as there's nothing forcing them to match

Maybe xargo CI could build a bunch of Xargo.toml files that should work? Plain libstd (maybe several feature combinations?), the one from this issue, ...
If you also set up a Travis cron job do re-run CI daily (like we did for miri), you will at least know when things break pretty quickly. (Currently I feel like miri is the most active CI testing the libstd-building part of xargo.^^)

[pkronstrom]

Hey, I am getting the same error. As a complete Rust newbie, I have no idea how to fix this manually. Any pointers?

rustc -V
rustc 1.22.0-nightly (0701b37d9 2017-09-18)
xargo -V
xargo 0.3.9 (7644989 2017-09-06)
cargo 0.23.0-nightly (8118b02ac 2017-09-14)

[Wallacoloo]

@pkronstrom downgrade your rust version:

$ rustup toolchain install nightly-2017-08-01
$ rustup default nightly-2017-08-01
$ rustup component add rust-src
$ xargo build

I first tried 2017-09-01 and it was still broken. 2017-08-01 seems to be working; likely there are more recent versions than that which will also work, if you need them.

[japaric]

downgrade your rust version:

Or install an older Xargo version. v0.3.8 works fine with a recent nightly. (cargo install xargo --vers 0.3.8 -f)

[RalfJung]

(a) continue using this assumption and patch rust-lang-nursery/compiler-builtins so that the version in its Cargo.toml matches the version in rust-lang/rust. Then we update the submodule in rust-lang/rust. Problem fixed (until someone changes either version ... as there's nothing forcing them to match)

I tried fixing the version number of the shim in rust-lang/rust#44742. Unfortunately, this doesn't work; there's code in bootstrap assuming the version numbers of all sorts of things to be 0.0.0.

[japaric]

@RalfJung What about doing it the other way around? That is changing the version of the compiler-builtins crate in the rust-lang-nursery/compiler-builtins repo to 0.0.0. Do you think that will work?

People keep asking about this issue and we are going to have a big influx of cortex-m users tomorrow and I'd rather not have them run into this. So, I have decide to yank the v0.3.9 release until this issue is fixed in master.

Xargo v0.3.8 can build std with jemalloc enabled with a recent nightly, in any case, so the .lock is not currently needed (I'm not entirely sure why it was needed in the first place -- you don't usually ship a .lock file with libraries and the std facade is comprised of libraries only -- did some dependency break semver?)

[RalfJung]

I don't have a strong opinion.

The reason I implemented the lock file stuff is that semver doesn't help in the presence of deny(warnings), which libstd uses. So, there was a library update deprecating stuff, and then libstd triggered deprecation warnings with the new version, and things failed to build with xargo. That situations has been resolved (libstd no longer uses the deprecated symbols), but it could happen again any time.

[RalfJung]

Nowadays compile-builtins is pulled in via crates.io, so this should not be a problem any more.