Hash mismatch with builtins.fetchUrl and .xz file

[lopsided98]

Since the most recent update of Nix unstable in nixpkgs, I have started having issues building fetchUrl derivations with files with the .xz extension. They are being decompressed even when the unpack attribute is not set, which causes them to have the wrong hash.

These commands demonstrate the problem:

$ nix-store --repair-path /nix/store/qdrjfapishmpp5f5i8dq22aynwrcl7aj-bootstrap-tools.tar.xz 
repairing outputs of '/nix/store/j4z5c8afgxclbdidfs5a9gpfvk7byjws-bootstrap-tools.tar.xz.drv'...
fixed-output derivation produced path '/nix/store/qdrjfapishmpp5f5i8dq22aynwrcl7aj-bootstrap-tools.tar.xz' with sha256 hash '1ndnr0lp0qzdkf6p7zp64hg7l2msgjpyqcmyq25y256h3ipinqy2' instead of the expected hash '05ayki2kak3i5lw97qidd5h9jv00dmlhx9h7l771bj331yamyqdn'
$ curl -L --output bootstrap-tools.tar.xz http://nixos-arm.dezgeg.me/bootstrap-2017-04-13-1f32d4b4/armv7l/bootstrap-tools.tar.xz
$ nix to-base32 --type sha256 $(sha256sum bootstrap-tools.tar.xz | cut -d ' ' -f1)
05ayki2kak3i5lw97qidd5h9jv00dmlhx9h7l771bj331yamyqdn
$ nix to-base32 --type sha256 $(xzcat bootstrap-tools.tar.xz | sha256sum | cut -d ' ' -f1)
1ndnr0lp0qzdkf6p7zp64hg7l2msgjpyqcmyq25y256h3ipinqy2

This bug was introduced by d3761f5, which creates a decompression sink for .xz files even when the unpack attribute is not set. The .xz logic used to be inside the unpack if statement, preventing this problem. I don't understand the code well enough to know the most elegant solution, but this issue can be trivially fixed by adding the unpack condition to the condition that defines whether a decompression sink is used.

[anka-213]

I can't upgrade nix because of this bug. Is there any way to circumvent it while upgrading to the version where it is fixed?

[edolstra]

Hm, why can't you upgrade?

[anka-213]

@edolstra Because nix depends on bootstrap-tools which refused to build because of this. I get the the error about mismatching hash for bootstrap-tools whenever I try to build anything on my raspberry pi.

However I managed to work around it by changing the bootstrap-tools.tar.xz.drv file to point to a server where I had a double-compressed copy of bootstrap-tools.tar.xz (bootstrap-tools.tar.xz.xz).

[dali99]

I'm having the same issue on my raspberry pi during initial installation

[endgame]

@dali99 if you are running armv7l images on your pi, there is a procedure to bootstrap a separate nix in the linked issue. Be prepared for long build times.

[JasonSwindle]

@anka-213 Got example of how you did this? I've been stuck on this error for too long. Thank you.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixosstag.fcio.net/t/nixos-rebuild-always-fails-on-armv7l-its-not-the-xz-bug/3576/1