[CA-drvs] Nix fails to remove directory trees

[fogti]

Describe the bug

I got the following error messages:

error: renaming '/nix/store/17zf7d3fh2ajwp9axs12lyh6khy8qdr6-bootstrap-stage0-binutils-wrapper-.drv.chroot/nix/store/a6szxq3w8fxha41z1m7621nmpvwvbh2p-bootstrap-stage0-binutils-wrapper-' to '/nix/store/s3dz0h5cvs1l5yialx32pl8y3z3i4nzg-bootstrap-stage0-binutils-wrapper-': Directory not empty
error (ignored): error: cannot unlink '/tmp/nix-build-xz-5.2.5.drv-0/xz-5.2.5/src/common': Directory not empty
error (ignored): error: cannot unlink '/tmp/nix-build-perl-5.32.1.drv-0/perl-5.32.1/cpan': Directory not empty
error (ignored): error: cannot unlink '/tmp/nix-build-gnum4-1.4.18.drv-0/m4-1.4.18': Directory not empty
error: renaming '/nix/store/phsmcx0dcil8fhbar22k4mdl3b9l446a-expand-response-params.drv.chroot/nix/store/1fa70l9kdbbg8y5hqr4y1svlkhk63ah0-expand-response-params' to '/nix/store/vyb8hzdlrm9x04ydwi5ijax67v7y7d1x-expand-response-params': Directory not empty

Steps To Reproduce

The above mentioned bootstrap-stage0-binutils-wrapper- directory was an aborted previous install, a non-valid store path, which just was still there. It should be possible to reproduce this by creating a fresh store, and build the stdenv with contentAddressedByDefault enabled when importing <nixpkgs> and -j1, terminating it mid-install, and restarting it. Sources shall be substituted from cache.nixos.org, but realizations won't be substituted from there, and cache.ng*.nixos.org won't substitute it bc of #4849.

Expected behavior

Nix should cleanup left-over directories. Currently, this does sometimes fail unexpectedly. Additionally, it seems as if it might try to rebuild something, and doesn't correctly deal with it when the destination is already present (due to CA-derivations).

nix-env --version output

nix-env (Nix) 2.4pre20210529_1f39092

Additional context

[fogti]

I suppose this bug wasn't present in Nix 2.4pre20210514_de9e43c and Nix 2.4pre20210506_db6ab75.
https://github.com/NixOS/nix/compare/de9e43c2ea8ac8752d42660c17a7b683682ef38b..1f390922d0e34382265205f6895c2b71c5a8e6f7

[thufschmitt]

Thanks for reporting this. I also encountered the issue (and I think with some slightly older Nix versions).

There might be two different issues there:

• The failed unlinking of the build directory seems (sort-of) benign in that the only consequence is that it will leave some garbage on disk
• The error with the existing and non-empty directory is more serious. I think there’s indeed an issue with Nix not properly detecting that the output path already exists under some mysterious conditions. Maybe there’s some locking missing somewhere.

[fogti]

I don't think it's directly locking related (at least not process-bound locking), as I thought I properly killed (SIGINT, then waiting for termination) the old nix process before starting a new one...

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/content-addressed-nix-call-for-testers/12881/118