Remote builds unconditionally triggering use-case-hack

[dezgeg]

On my ARM builder's master machine (an old x86_64 NixOS laptop) some paths have the case hack applied without any involvement of MacOS machines:

root@master $ find /nix/store/gwgdvvkha1rhbd7ca6rlw43g9ml3ls00-linux-headers-4.15 -name '*case*hack*'
/nix/store/gwgdvvkha1rhbd7ca6rlw43g9ml3ls00-linux-headers-4.15/include/linux/netfilter/xt_connmark.h~nix~case~hack~1
/nix/store/gwgdvvkha1rhbd7ca6rlw43g9ml3ls00-linux-headers-4.15/include/linux/netfilter/xt_dscp.h~nix~case~hack~1

On the slave (an ARMv7 NixOS board) the path is fine, i.e. the case hack suffixes don't exist.

If I gc the path on master and realize the derivation again (i.e. nix-store -r /nix/store/j06agj9f3s3v44xyh27048wrmh0w09iw-linux-headers-4.15.drv) the problem is reproduced, i.e. the case hack suffixes are still there. If I copy-closure the path from the slave, the path is fine on the master as well.

Both have Nix 2.0.

I suppose the real problem is:

 18 struct CaseHackSetting : public BaseSetting<bool>
 19 {
...
 29     void set(const std::string & str) override
 30     {
 31         BaseSetting<bool>::set(str);
 32         nix::useCaseHack = true;
 33     }
 34 };

and somewhere in the remote build path the above gets triggered even if the value is set to false, as I'm guessing from this use-case-hack\0\0\0\5\0\0\0\0\0\0\0false\0\ which is written to the /nix/store/lg0s7z1bvj2chimz3a4qax3y8ssw45m3-nix-2.0/libexec/nix/build-remote process via a pipe:

29913 write(12, "\1\0\0\0\0\0\0\0\34\0\0\0\0\0\0\0allow-import-from-derivation\0\0\0\0\4\0\0\0\0\0\0\0true\0\0\0\0\1\0\0\0\0\0\0\0\24\0\0\0\0\0\0\0allow-new-privileges\0\0\0\0\5\0\0\0\0\0\0\0false\0\0\0\1\0\0\0\0\0\0\0*\0\0\0\0\0\0\0allow-unsafe-native-code-during-evaluation\0\0\0\0\0\0\5\0\0\0\0\0\0\0false\0\0\0\1\0\0\0\0\0\0\0\30\0\0\0\0\0\0\0allowed-impure-host-deps\0\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\f\0\0\0\0\0\0\0allowed-uris\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\r\0\0\0\0\0\0\0allowed-users\0\0\0\1\0\0\0\0\0\0\0*\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\23\0\0\0\0\0\0\0auto-optimise-store\0\0\0\0\0\5\0\0\0\0\0\0\0false\0\0\0\1\0\0\0\0\0\0\0\n\0\0\0\0\0\0\0build-hook\0\0\0\0\0\0L\0\0\0\0\0\0\0/nix/store/lg0s7z1bvj2chimz3a4qax3y8ssw45m3-nix-2.0/libexec/nix/build-remote\0\0\0\0\1\0\0\0\0\0\0\0\23\0\0\0\0\0\0\0build-poll-interval\0\0\0\0\0\1\0\0\0\0\0\0\0005\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\21\0\0\0\0\0\0\0build-users-group\0\0\0\0\0\0\0\6\0\0\0\0\0\0\0nixbld\0\0\1\0\0\0\0\0\0\0\10\0\0\0\0\0\0\0builders\22\0\0\0\0\0\0\0@/etc/nix/machines\0\0\0\0\0\0\1\0\0\0\0\0\0\0\30\0\0\0\0\0\0\0builders-use-substitutes\5\0\0\0\0\0\0\0false\0\0\0\1\0\0\0\0\0\0\0\22\0\0\0\0\0\0\0compress-build-log\0\0\0\0\0\0\4\0\0\0\0\0\0\0true\0\0\0\0\1\0\0\0\0\0\0\0\17\0\0\0\0\0\0\0connect-timeout\0\1\0\0\0\0\0\0\0000\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\5\0\0\0\0\0\0\0cores\0\0\0\1\0\0\0\0\0\0\0000\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\t\0\0\0\0\0\0\0diff-hook\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\23\0\0\0\0\0\0\0enforce-determinism\0\0\0\0\0\4\0\0\0\0\0\0\0true\0\0\0\0\1\0\0\0\0\0\0\0\23\0\0\0\0\0\0\0extra-sandbox-paths\0\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\22\0\0\0\0\0\0\0extra-substituters\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\10\0\0\0\0\0\0\0fallback\5\0\0\0\0\0\0\0false\0\0\0\1\0\0\0\0\0\0\0\17\0\0\0\0\0\0\0filter-syscalls\0\4\0\0\0\0\0\0\0true\0\0\0\0\1\0\0\0\0\0\0\0\16\0\0\0\0\0\0\0fsync-metadata\0\0\4\0\0\0\0\0\0\0true\0\0\0\0\1\0\0\0\0\0\0\0\25\0\0\0\0\0\0\0gc-check-reachability\0\0\0\5\0\0\0\0\0\0\0false\0\0\0\1\0\0\0\0\0\0\0\21\0\0\0\0\0\0\0gc-reserved-space\0\0\0\0\0\0\0\7\0\0\0\0\0\0\0008388608\0\1\0\0\0\0\0\0\0\16\0\0\0\0\0\0\0hashed-mirrors\0\0\32\0\0\0\0\0\0\0http://tarballs.nixos.org/\0\0\0\0\0\0\1\0\0\0\0\0\0\0\20\0\0\0\0\0\0\0http-connections\2\0\0\0\0\0\0\00025\0\0\0\0\0\0\1\0\0\0\0\0\0\0\5\0\0\0\0\0\0\0http2\0\0\0\4\0\0\0\0\0\0\0true\0\0\0\0\1\0\0\0\0\0\0\0\24\0\0\0\0\0\0\0impersonate-linux-26\0\0\0\0\5\0\0\0\0\0\0\0false\0\0\0\1\0\0\0\0\0\0\0\16\0\0\0\0\0\0\0keep-build-log\0\0\4\0\0\0\0\0\0\0true\0\0\0\0\1\0\0\0\0\0\0\0\20\0\0\0\0\0\0\0keep-derivations\4\0\0\0\0\0\0\0true\0\0\0\0\1\0\0\0\0\0\0\0\24\0\0\0\0\0\0\0keep-env-derivations\0\0\0\0\5\0\0\0\0\0\0\0false\0\0\0\1\0\0\0\0\0\0\0\v\0\0\0\0\0\0\0keep-failed\0\0\0\0\0\5\0\0\0\0\0\0\0false\0\0\0\1\0\0\0\0\0\0\0\n\0\0\0\0\0\0\0keep-going\0\0\0\0\0\0\5\0\0\0\0\0\0\0false\0\0\0\1\0\0\0\0\0\0\0\f\0\0\0\0\0\0\0keep-outputs\0\0\0\0\5\0\0\0\0\0\0\0false\0\0\0\1\0\0\0\0\0\0\0\22\0\0\0\0\0\0\0max-build-log-size\0\0\0\0\0\0\1\0\0\0\0\0\0\0000\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\10\0\0\0\0\0\0\0max-free\24\0\0\0\0\0\0\00018446744073709551615\0\0\0\0\1\0\0\0\0\0\0\0\10\0\0\0\0\0\0\0max-jobs\1\0\0\0\0\0\0\0004\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\17\0\0\0\0\0\0\0max-silent-time\0\1\0\0\0\0\0\0\0000\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\10\0\0\0\0\0\0\0min-free\1\0\0\0\0\0\0\0000\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\n\0\0\0\0\0\0\0netrc-file\0\0\0\0\0\0\16\0\0\0\0\0\0\0/etc/nix/netrc\0\0\1\0\0\0\0\0\0\0\f\0\0\0\0\0\0\0plugin-files\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\16\0\0\0\0\0\0\0pre-build-hook\0\0\0\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\r\0\0\0\0\0\0\0print-missing\0\0\0\4\0\0\0\0\0\0\0true\0\0\0\0\1\0\0\0\0\0\0\0\t\0\0\0\0\0\0\0pure-eval\0\0\0\0\0\0\0\5\0\0\0\0\0\0\0false\0\0\0\1\0\0\0\0\0\0\0\6\0\0\0\0\0\0\0repeat\0\0\1\0\0\0\0\0\0\0000\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\f\0\0\0\0\0\0\0require-sigs\0\0\0\0\4\0\0\0\0\0\0\0true\0\0\0\0\1\0\0\0\0\0\0\0\r\0\0\0\0\0\0\0restrict-eval\0\0\0\5\0\0\0\0\0\0\0false\0\0\0\1\0\0\0\0\0\0\0\r\0\0\0\0\0\0\0run-diff-hook\0\0\0\5\0\0\0\0\0\0\0false\0\0\0\1\0\0\0\0\0\0\0\7\0\0\0\0\0\0\0sandbox\0\5\0\0\0\0\0\0\0false\0\0\0\1\0\0\0\0\0\0\0\21\0\0\0\0\0\0\0sandbox-build-dir\0\0\0\0\0\0\0\6\0\0\0\0\0\0\0/build\0\0\1\0\0\0\0\0\0\0\24\0\0\0\0\0\0\0sandbox-dev-shm-size\0\0\0\0\3\0\0\0\0\0\0\00050%\0\0\0\0\0\1\0\0\0\0\0\0\0\r\0\0\0\0\0\0\0sandbox-paths\0\0\0N\0\0\0\0\0\0\0/bin/sh=/nix/store/9h8hqpns8ingdisqgybzdfv9dg8f80kv-busybox-1.28.1/bin/busybox\0\0\1\0\0\0\0\0\0\0\20\0\0\0\0\0\0\0secret-key-files\0\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\n\0\0\0\0\0\0\0show-trace\0\0\0\0\0\0\5\0\0\0\0\0\0\0false\0\0\0\1\0\0\0\0\0\0\0\5\0\0\0\0\0\0\0store\0\0\0\4\0\0\0\0\0\0\0auto\0\0\0\0\1\0\0\0\0\0\0\0\n\0\0\0\0\0\0\0substitute\0\0\0\0\0\0\5\0\0\0\0\0\0\0false\0\0\0\1\0\0\0\0\0\0\0\f\0\0\0\0\0\0\0substituters\0\0\0\0:\0\0\0\0\0\0\0https://cache.nixos.org http://nixos-arm.dezgeg.me/channel\0\0\0\0\0\0\1\0\0\0\0\0\0\0\27\0\0\0\0\0\0\0sync-before-registering\0\5\0\0\0\0\0\0\0false\0\0\0\1\0\0\0\0\0\0\0\6\0\0\0\0\0\0\0system\0\0\f\0\0\0\0\0\0\0x86_64-linux\0\0\0\0\1\0\0\0\0\0\0\0\v\0\0\0\0\0\0\0tarball-ttl\0\0\0\0\0\4\0\0\0\0\0\0\0003600\0\0\0\0\1\0\0\0\0\0\0\0\7\0\0\0\0\0\0\0timeout\0\1\0\0\0\0\0\0\0000\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\23\0\0\0\0\0\0\0trusted-public-keys\0\0\0\0\0\202\0\0\0\0\0\0\0nixos-arm.dezgeg.me-1:xBaUKS3n17BZPKeyxL4JfbTqECsT+ysbDJz29kLFRW0=% cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=\0\0\0\0\0\0\1\0\0\0\0\0\0\0\24\0\0\0\0\0\0\0trusted-substituters\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\r\0\0\0\0\0\0\0trusted-users\0\0\0\10\0\0\0\0\0\0\0tmtynkky\1\0\0\0\0\0\0\0\r\0\0\0\0\0\0\0use-case-hack\0\0\0\5\0\0\0\0\0\0\0false\0\0\0\1\0\0\0\0\0\0\0\16\0\0\0\0\0\0\0use-sqlite-wal\0\0\4\0\0\0\0\0\0\0true\0\0\0\0\1\0\0\0\0\0\0\0\21\0\0\0\0\0\0\0user-agent-suffix\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\3\0\0\0\0\0\0\0try\0\0\0\0\0\1\0\0\0\0\0\0\0\f\0\0\0\0\0\0\0armv7l-linux\0\0\0\0B\0\0\0\0\0\0\0/nix/store/j06agj9f3s3v44xyh27048wrmh0w09iw-linux-headers-4.15.drv\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 3800 <unfinished ..

[coreyoconnor]

The patch at ac19837 contains a change worth considering: always remove the "nixcasehack" from source files when building a NAR. Don't only check if use case hack is true.

I'm using this to recover from the corrupted nix store by, essentially, transferring the NARs to the same computer. (A weird effect of having nix-serve serve the store being repaired!)