Incorrect dev-dependency feature resolution

[farcaller]

If same crate is used in both dependencies and dev-dependencies with the only change of features, that features leak from dev-dependencies to dependencies.

Here's the smallest test case I managed to generate.

Root crate Cargo.toml:

[package]
name = "test"
version = "0.1.0"
authors = ["Vladimir Pouzanov <farcaller@gmail.com>"]

[lib]
name = "test"
crate-type = ["lib"]

[target.thumbv6-none-eabi.dependencies.core]
git = "https://github.com/hackndev/rust-libcore"

[dev-dependencies]
expectest = "*"

[dev-dependencies.dep]
path = "./dep"
features = ["replayer"]

[dependencies.dep]
path = "./dep"

this crate uses expectest for testing only and a dep crate for either testing or runtime build. It is expected to be a cross-compiled staticlib, so it does not depend on core (neither is dep).

Dep is defined like this:

[package]

name = "dep"
version = "1.0.0"
authors = ["Vladimir Pouzanov <farcaller@gmail.com>"]

[lib]
name = "dep"
path = "lib.rs"
crate-type = ["rlib"]

[features]
replayer = ["expectest"]

[target.thumbv6-none-eabi.dependencies.core]
git = "https://github.com/hackndev/rust-libcore"

[dependencies.expectest]
optional = true

If the replayer feature is used, this crate is being built for test, thus requiring expectest in runtime. Otherwise this crate is std-less as well.

When cross-compiled, the expected outcome is to have test and dep crates being cross-built, where dep is being built without expectest. This is not the case:

% cargo build --target=thumbv6-none-eabi --verbose -j1 --lib
   Compiling rustc-serialize v0.3.15
     Running `rustc /Users/farcaller/.cargo/registry/src/github.com-0a35038f75765ae4/rustc-serialize-0.3.15/src/lib.rs --crate-name rustc_serialize --crate-type lib -g -C metadata=c1e8163a38ed3d54 -C extra-filename=-c1e8163a38ed3d54 --out-dir /Users/farcaller/temp/bug/test/target/thumbv6-none-eabi/debug/deps --emit=dep-info,link --target thumbv6-none-eabi -L dependency=/Users/farcaller/temp/bug/test/target/thumbv6-none-eabi/debug/deps -L dependency=/Users/farcaller/temp/bug/test/target/thumbv6-none-eabi/debug/deps -Awarnings`
/Users/farcaller/.cargo/registry/src/github.com-0a35038f75765ae4/rustc-serialize-0.3.15/src/lib.rs:1:1: 1:1 error: can't find crate for `std`
/Users/farcaller/.cargo/registry/src/github.com-0a35038f75765ae4/rustc-serialize-0.3.15/src/lib.rs:1 // Copyright 2012-2014 The Rust Project Developers. See the COPYRIGHT

rustc_serialize is being pulled in as a test dependency through test -> dep -> expectest chain and fails to compile in cross environment.

[farcaller]

I traced it down to https://github.com/rust-lang/cargo/blob/master/src/cargo/ops/cargo_compile.rs#L152 which seem to be the problem.

This simple patch works for me (both cargo build and cargo test work as expected):

diff --git a/src/cargo/ops/cargo_compile.rs b/src/cargo/ops/cargo_compile.rs
index a927208..5d9ff3e 100644
--- a/src/cargo/ops/cargo_compile.rs
+++ b/src/cargo/ops/cargo_compile.rs
@@ -149,7 +149,8 @@ pub fn compile_pkg<'a>(package: &Package,
         let platform = target.as_ref().map(|e| &e[..]).or(Some(&rustc_host[..]));

         let method = Method::Required{
-            dev_deps: true, // TODO: remove this option?
+            dev_deps: options.mode == CompileMode::Test ||
+                options.mode == CompileMode::Bench,
             features: &features,
             uses_default_features: !no_default_features,
             target_platform: platform};

[farcaller]

After tinkering around the source it seems that it's more of an architectural problem.

ops::resolve_with_previous resolves dependencies for a "Package", so all targets of that package will have same set of dependencies, both library and examples.

This makes me come to a conclusion that the only difference between dependencies and dev-dependencies is propagation factor to other crates.

To sum up why am I here at this point: cargo build will pull in dev-dependencies even if it builds only the lib crate. It makes it very non-trivial to provide nice unit-tests for a cross-compiled no-std crate.

So far all my attempts to unbreak the examples (building them with dev-deps) failed and it looks like the change is beyond my understanding of cargo source so I'm closing the PR and bringing the discussion back here in hopes that someone else will give me a hint on how to fix this or provide a better PR.

[farcaller]

For the reference, here's a unit test exposing the problem:

test!(dev_dependencies_dont_leak_features {
    let p = project("foo")
        .file("Cargo.toml", r#"
            [package]
            name = "foo"
            version = "0.1.0"
            authors = []

            [dependencies.a]
            path = "a"

            [dev-dependencies.a]
            path = "a"
            features = ["badfeature"]
        "#)
        .file("src/main.rs", r#"
            extern crate a;
            pub fn main() { a::a(); }
        "#)
        .file("a/Cargo.toml", r#"
            [package]
            name = "a"
            version = "0.1.0"
            authors = []

            [features]
            badfeature = []
        "#)
        .file("a/src/lib.rs", r#"
            #[cfg(feature = "badfeature")]
            pub fn a() { panic!(); }

            #[cfg(not(feature = "badfeature"))]
            pub fn a() { }
        "#);

    assert_that(p.cargo_process("run"), execs().with_status(0));
});

[alexcrichton]

Hm, upon thinking on this again I don't think there's actually any alternative here. Architecturally Cargo will union all features when compiling a crate, so even if we kept cargo build and cargo test apart in terms of activating the features of dev-dependencies, you'd have a situation like this:

• For cargo build the feature would not be activated and the library would be compiled as usual.
• For cargo test the upstream dependency would be recompiled with the new feature, and the library would be compiled against this new copy of the library for integration tests and such.

This means that even if we were to draw the distinction between cargo build and cargo test you'd still have to write a library ready to link against both versions of the upstream dependency.

I'll also clarify that this is a very specific problem, it only shows up when:

• The exact same dependency is used as both a dev and non-dev dependency.
• The dev-dependency version activates a feature

All in all I'm starting to think that this is working as intended rather than a bug.

[farcaller]

Here's my practical concern.

zinc pulls in volatile_cell in either runtime or test. It also pulls in expectest for testing.

volatile_cell supports mocking if built with replayer feature. If it's enabled, expectest is pulled in too.

I can't change expectest to dev-dependency for volatile_cell as it doesn't propagate when I build zinc crate for tests and I can't use it as non-dev dependency now as building zinc for release pulls in expectest and its dependencies all way down to libstd.

Is it a wrong use of cargo or unsupported usecase? It would be sad for us to drop cargo support again, but we really need to have both cross-builds and unit tests done in a reasonable manner (i.e. without ./configure generating the Cargo.toml).

[alexcrichton]

Could you put the activation of replayer in volatile_cell behind a feature of the top-level crate itself? That way cargo test would run unit tests and cargo test --feature replayer would just run "more" unit tests.

[apoelstra]

I ran into this bug this week. I've got a feature qa on a library which adds some extra required fields to my program's configuration file. These extra fields are used by the integration tests as configuration for the test harness; they have no purpose for ordinary users of the library.

[alexcrichton]

@apoelstra the bug here should just be that when building the crate itself the feature leaks into the dependency, but when you crate is itself used as a dependency it shouldn't activate the features activated as part of the dev-dependencies. Does this differ from what you're seeing though?

[apoelstra]

@alexcrichton The crate that depends on my library is a binary; it's never used as a dependency. It's when I'm building the binary that I'm having problems.

[alexcrichton]

Ah yes that'd do it. Can you elaborate on the problems you're seeing though? I understand that the dependence is built with the extra features during a normal cargo build, but I'm curious how this manifests itself as a problem down the road