Nix 2.4 regression, missing paths when evaluating inside the sandbox.

[roberth]

Describe the bug

While updating the nix package in Nixpkgs, NixOS/nixpkgs#144197 (comment)

The nixpkgs-release-checks derivation invokes a few nix-* commands inside the sandbox. This used to work.

The command that breaks inside the sandbox is:

        nix-env -f $src \
            --show-trace --argstr system "$platform" \
            --arg config '{ allowAliases = false; }' \
            -qa --drv-path --system-filter \* --system \
            "''${opts[@]}" 2>&1 >/dev/null | tee eval-warnings.log

Steps To Reproduce

1. Rebase nixpkgs PR 144197 onto 88d6a4f9ef8ec7aa00f6c810b3fe244f991eaf00 (base of the PR at the time of discovery)
2. nix-build pkgs/top-level/release.nix -A tarball.nixpkgs-basic-release-checks

Expected behavior

Build success.

nix-env --version output

My NixOS nix-daemon is
nix-env (Nix) 2.4pre20210707_02dd6bb

The Nix in the sandbox is the first 2.4 release.

Additional context

I've tried a different store location, but that did not make a difference; see diff.

--- a/pkgs/top-level/nixpkgs-basic-release-checks.nix
+++ b/pkgs/top-level/nixpkgs-basic-release-checks.nix
@@ -5,8 +5,8 @@ pkgs.runCommand "nixpkgs-release-checks" { src = nixpkgs; buildInputs = [nix]; }
 
     export NIX_STATE_DIR=$TMPDIR
     export NIX_PATH=nixpkgs=$TMPDIR/barf.nix
-    opts=(--option build-users-group "")
-    nix-store --init
+    opts=(--option build-users-group "" --option store "$TMPDIR/.physical-store-location")
+    nix-store --init "''${opts[@]}"
 
     echo 'abort "Illegal use of <nixpkgs> in Nixpkgs."' > $TMPDIR/barf.nix

Console log (regardless of diff above):

nixpkgs]$ nix-build pkgs/top-level/release.nix -A tarball.nixpkgs-basic-release-checks 
this derivation will be built:
  /nix/store/bgm2hvq921fafirbqw7svjnvmnvy7kf6-nixpkgs-release-checks.drv
building '/nix/store/bgm2hvq921fafirbqw7svjnvmnvy7kf6-nixpkgs-release-checks.drv'...
warning: you did not specify '--add-root'; the result might be removed by the garbage collector
warning: you did not specify '--add-root'; the result might be removed by the garbage collector
checking Nixpkgs on x86_64-linux
error: path '/nix/store/hdr4yj37klan2w6dlif6hpbfazcxxb48-source' is not valid

       … while adding path '/nix/store/hdr4yj37klan2w6dlif6hpbfazcxxb48-source/pkgs/tools/misc/fx_cast'

       at /nix/store/hdr4yj37klan2w6dlif6hpbfazcxxb48-source/pkgs/build-support/nix-gitignore/default.nix:162:5:

          161|   gitignoreFilterSourcePure = filter: patterns: root:
          162|     filterSource (gitignoreFilterPure filter patterns root) root;
             |     ^
          163|

       … while evaluating 'gitignoreFilterSourcePure'

       at /nix/store/hdr4yj37klan2w6dlif6hpbfazcxxb48-source/pkgs/build-support/nix-gitignore/default.nix:161:49:

          160|
          161|   gitignoreFilterSourcePure = filter: patterns: root:
             |                                                 ^
          162|     filterSource (gitignoreFilterPure filter patterns root) root;

       … from call site

       at /nix/store/hdr4yj37klan2w6dlif6hpbfazcxxb48-source/pkgs/tools/misc/fx_cast/node-packages.nix:1697:13:

         1696|       name = args.name + "-package-json";
         1697|       src = nix-gitignore.gitignoreSourcePure [
             |             ^
         1698|         "*"

       … while evaluating the attribute 'src' of the derivation 'fx_cast_bridge-package-json'

       at /nix/store/hdr4yj37klan2w6dlif6hpbfazcxxb48-source/pkgs/stdenv/generic/make-derivation.nix:205:7:

          204|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          205|       name =
             |       ^
          206|         let

       … while evaluating the attribute 'installPhase' of the derivation 'node-dependencies-fx_cast_bridge-0.1.2'

       at /nix/store/hdr4yj37klan2w6dlif6hpbfazcxxb48-source/pkgs/stdenv/generic/make-derivation.nix:205:7:

          204|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          205|       name =
             |       ^
          206|         let

       … while evaluating the attribute 'buildPhase' of the derivation 'fx_cast_bridge-0.1.2'

       at /nix/store/hdr4yj37klan2w6dlif6hpbfazcxxb48-source/pkgs/stdenv/generic/make-derivation.nix:205:7:

          204|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          205|       name =
             |       ^
          206|         let

       … while querying the derivation named 'fx_cast_bridge-0.1.2'
error: builder for '/nix/store/bgm2hvq921fafirbqw7svjnvmnvy7kf6-nixpkgs-release-checks.drv' failed with exit code 1;
       last 10 log lines:
       >        … while evaluating the attribute 'buildPhase' of the derivation 'fx_cast_bridge-0.1.2'
       >
       >        at /nix/store/hdr4yj37klan2w6dlif6hpbfazcxxb48-source/pkgs/stdenv/generic/make-derivation.nix:205:7:
       >
       >           204|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
       >           205|       name =
       >              |       ^
       >           206|         let
       >
       >        … while querying the derivation named 'fx_cast_bridge-0.1.2'
       For full logs, run 'nix log /nix/store/bgm2hvq921fafirbqw7svjnvmnvy7kf6-nixpkgs-release-checks.drv'.

[pennae]

also reproducible in a plain shell with

src=$(nix-store --add "$PWD") # in a nixpkgs checkout
export NIX_STATE_DIR=$(mktemp -d) # does not fail if this isn't set
nix-env -f $src --drv-path -qa >/dev/null # but not without --drv-path

[edolstra]

The new dummy store (--store dummy://) may be useful if you want to do read-only evaluation.

BTW, the error

error: path '/nix/store/hdr4yj37klan2w6dlif6hpbfazcxxb48-source' is not valid

suggests that there is some IFD going on, or at least import-from-a-path-copied-to-the-store, which we should avoid in Nixpkgs...

[pennae]

suggests that there is some IFD going on

not just IFD, that error message (though maybe not the exact same error) also happens when evaluating builtins.filterSource (_:_: true) $src with NIX_STATE_DIR set to an empty directory. (or does that count as IFD? thought it doesn't, but now i'm not so sure any more)

[thiagokokada]

Another example that seems to be causing problems:

outPath = builtins.path { inherit filter name; path = origSrc; };

You can look at this and other examples on NixOS/nixpkgs#144197 (comment), where I was trying to "fix" the eval errors I was getting by simplifying them, until I eventually gave up because it simply went to the next error: path '/nix/store/something' is not valid.

[thiagokokada]

It seems most (all?) errors are coming from builtins.filterSource usage. nix-gitignore uses it here and here, while builtins.path uses it with filter parameter.

[thufschmitt]

I think the issue is that $src is used as if it were a “normal” filesystem path (/foo/bar), whereas Nix recognizes it as a store path, and thus wants to make sure that it’s a valid path (not sure why/when this restriction was added, but it makes sense in itself as the store directory can contain any kind of garbage, but that garbage is just some internal implementation detail and shouldn’t be accessed by anyone). But because NIX_STATE_DIR is overriden (and because the sandbox doesn’t have access to the real state dir anyways), the path isn’t recognized as valid, hence the error.

In hindsight, this makes a lot of sense (forgetting about the fact that it’s a breaking change).

I’m not sure what the “proper” fix should be, but I think that a decent solution would be to set NIX_STORE_DIR to point to something else than /nix/store (something like export NIX_STORE_DIR=$PWD/.store). Unless the result of the evaluation somehow hardcodes /nix/store (which it certainly shouldn’t, and it’s probably nice to also check that), that should do exactly what you want.

Maybe that would also be a good use-case for recursive-nix, but since it’s still experimental so far that’s out of reach for nixpkgs.

[lovesegfault]

I think the issue is that $src is used as if it were a “normal” filesystem path (/foo/bar), whereas Nix recognizes it as a store path, and thus wants to make sure that it’s a valid path (not sure why/when this restriction was added, but it makes sense in itself as the store directory can contain any kind of garbage, but that garbage is just some internal implementation detail and shouldn’t be accessed by anyone). But because NIX_STATE_DIR is overriden (and because the sandbox doesn’t have access to the real state dir anyways), the path isn’t recognized as valid, hence the error.

In hindsight, this makes a lot of sense (forgetting about the fact that it’s a breaking change).

I’m not sure what the “proper” fix should be, but I think that a decent solution would be to set NIX_STORE_DIR to point to something else than /nix/store (something like export NIX_STORE_DIR=$PWD/.store). Unless the result of the evaluation somehow hardcodes /nix/store (which it certainly shouldn’t, and it’s probably nice to also check that), that should do exactly what you want.

Maybe that would also be a good use-case for recursive-nix, but since it’s still experimental so far that’s out of reach for nixpkgs.

Indeed setting NIX_STORE_DIR works around the issue in the test! This should unblock the move to 2.4 in nixpkgs, hopefully :)

[fogti]

I would like to point out that it appears that with #6714 applied (or at least that's what I was able to infer from the error message), the workaround/fix using NIX_STORE_DIR ceases to work: #6732

[thufschmitt]

Urgh indeed. I guess mkdir "$NIX_STATE_DIR" is enough to fix it, but that's quite dirty nonetheless :/