buildRustPackage: found duplicate version of package

[ljli]

I wanted to upgrade rustup to 1.6 now that we have a recent rust compiler again thanks to #30088.
Unfortunately I got an error, the log tail is below.
Rustup uses a [replace] section for mio in its Cargo.toml, so that is probably the cause for the error.
The error output itself is from cargo-vendor. I would report it there but I'm not sure why it can't handle this, maybe there is some good reason and we should use it in a different way.
There shouldn't be any trouble to reproduce this, updating the hash and version for the rustup derivation
should do.
I suspect this is a general problem with cargo packages that use [replace], hence the issue name.

$ nix-build '<nixpkgs-work>' -A rustup --no-out-link
...
 Downloading aho-corasick v0.5.3
 Downloading lzma-sys v0.1.8
 Downloading foreign-types v0.2.0
 Downloading gcc v0.3.53
error: failed to sync

Caused by:
  found duplicate version of package `mio v0.6.10` vendored from two sources:

	source 1: https://github.com/carllerche/mio#6871f83a
	source 2: registry https://github.com/rust-lang/crates.io-index
builder for ‘/nix/store/ycmixz6k8djya309dl67x4px97qr595c-rustup-1.6.0-vendor.drv’ failed with exit code 101
cannot build derivation ‘/nix/store/sq4ci1azfqg1jnmsj6zdjcjaj89cdbjk-rustup-1.6.0.drv’: 1 dependencies couldn't be built
error: build of ‘/nix/store/sq4ci1azfqg1jnmsj6zdjcjaj89cdbjk-rustup-1.6.0.drv’ failed

[wizeman]

cc @zimbatm

[zimbatm]

I did the packaging but actually don't have much insight on how the internals are working. The cargo-vendor author might be able to tell us why the [replace] directive is not overriding the crates.io entry.

[alex-ameen]

Hey I have run into this issue trying to build a package for "spotifyd".
I get duplicate versions of 'rust-crypto', which is the only [replace] directive in the project.

Did you ever find a solution?

[ljli]

FTR, cargo-vendor has a flag (--no-merge-sources) now which solves the root problem. If buildRustPackage gets adapted, crates like spotifyd should work.

[bobvanderlinden]

I was also working on a package for spotifyd (see #47172), but just like @BadDecisionsAlex I ran into this issue. @ljli I couldn't find the no-merge-sources in cargo-vendor. I tried adding --no-merge-sources to the cargoBuildFlags, but that still resulted in the same error. Any ideas?

Also, I was able to build spotifyd inside nix-shell:

nix-shell -p cargo -p alsaLib -p libpulseaudio -p pkgconfig -p openssl -p dbus
cargo build --release --features pulseaudio_backend

[ljli]

The flag --no-merge-sources is available in the current cargo-vendor master branch. So first of all we need to update it in nixpkgs.
The place to pass the flag would then be here:

nixpkgs/pkgs/build-support/rust/fetchcargo.nix

Line 45 in fcde178

cargo vendor $out | cargo-vendor-normalise > config

I'll try to have a look tomorrow.

[ljli]

My first attempt to hook up --no-merge-sources is here:
https://github.com/ljli/nixpkgs/tree/cargo-vendor-multi-sources

I think something along those lines should work, it is based on staging because of #46362.
Unfortunately I hit the following now:

$ nix build nixpkgs-work.spotifyd
error: path '/nix/store/kyvsqxp7cz3f0dhf1l7ly1vw9208wn88-spotifyd-0.2.2-vendor' is not valid

I have no clue why, maybe this is a local problem so you are welcome to try the branch and/or pick it up from there, otherwise I'll investigate after staging got merged.

[boomshroom]

Could you perhaps submit a pull request for that? I'm running into the same problem.

[zimbatm]

A bunch of things have changed on the rust packaging side which probably fix this issue. Ping me if I should re-open.

[bqv]

@zimbatm I think this issue is still quite relevant. See:

rustPlatform.buildRustPackage {
            pname = "weechat-discord";
            version = "test";

            src = fetchFromGitHub {
              owner = "terminal-discord";
              repo = "weechat-discord";
              rev = "e09bac671b14c389f9909d8393ab76a575c5df73";
              sha256 = "0n1681vn0w2a4zjfmaimbbjsfpc5aflp9rr5w7j3zh1mp87wnxdj";
            };

            cargoSha256 = "0000000000000000000000000000000000000000000000000000";

            buildInputs = [
            ];
          }

Which for me, fails with:

  (...)
  Downloaded winapi-i686-pc-windows-gnu v0.4.0
error: failed to sync

Caused by:
  found duplicate version of package `parking_lot v0.9.0` vendored from two sources:

        source 1: https://github.com/terminal-discord/parking_lot?rev=046a171#046a1714
        source 2: registry `https://github.com/rust-lang/crates.io-index`
Traceback (most recent call last):
  File "/nix/store/14qgqb8avba5fbpsdwmnys49h2k8ljmk-cargo-vendor-normalise/bin/.cargo-vendor-normalise-wrapped", line 42, in <module>
    main()
  File "/nix/store/14qgqb8avba5fbpsdwmnys49h2k8ljmk-cargo-vendor-normalise/bin/.cargo-vendor-normalise-wrapped", line 17, in main
    assert list(data.keys()) == ["source"]
AssertionError
builder for '/nix/store/ccwiz2jm9kvfgc9qk3gddxpy7m6c1par-weechat-discord-beta-vendor.drv' failed with exit code 1
cannot build derivation '/nix/store/gg7fwmp6yr5xh83w4idxla65nxa2ccqp-weechat-discord-beta.drv': 1 dependencies couldn't be built
error: build of '/nix/store/gg7fwmp6yr5xh83w4idxla65nxa2ccqp-weechat-discord-beta.drv' failed