dockerTools: populate the nix database of the container nix store

[nlewo]

Currently, the contents closure is copied to the layer but there is no
nix database initialization. If pkgs.nix is added in the contents,
nix-store doesn't work because there is no nix database.

From the contents of the layer, this commit generates and loads the
database in the nix store of the container. This only works if there
is no parent layer that already have a nix store (to support several
nix layers, we would have to merge nix databases of parent layers).

We also add an example to play with the nix store inside the
container. Note it seems more is a missing dependency of the nix
package!

Motivation for this change

Be able to use nix in a container

Things done

• Tested using sandboxing
  (nix.useSandbox on NixOS,
  or option build-use-sandbox in nix.conf
  on non-NixOS)
• Built on platform(s)
  • NixOS
  • macOS
  • Linux
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Fits CONTRIBUTING.md.

---

[nlewo]

cc @lo1tuma @NeQuissimus @globin

[puffnfresh]

This should be optional or outside of dockerTools. We don't use Nix inside of containers and do not want to increase our image sizes with a Nix DB.

[nlewo]

I use nix inside the container because my container is a hydra slave. It could also be useful to be able to use nix inside the container for debugging purpose.
But I agree this increases the container size and this could be optional (or maybe, we could enable it by default and let the user disable it easily).

[nlewo]

@puffnfresh I added the option populateNixDb to dockerTools.buildImage and set it to false by default.

[puffnfresh]

@nlewo can you write this as a function which creates a new layer instead of modifying mkPureLayer?

[globin]

I'd definitely prefer making this opt-in

[nlewo]

@puffnfresh I think it's not trivial to add a layer, and I'm not sure it's a good idea to have a layer with a nix database without the store associated to that database.
Since the current patch doesn't change the actual behavior, but allow us to use nix inside the container, could we discuss your proposition in another PR?

Note also, we currently publish on the docker hub a docker image (nixos/nix) built with a dockerfile based on alpine. With this patch, we are going to generate this image with our tools :)

[nlewo]

ping

[puffnfresh]

@nlewo isn't this enough?

with import <nixpkgs> { };

let
  nixRegistration = contents: runCommand "nix-registration" {
    buildInputs = [ nixUnstable perl ];
    # For obtaining the closure of `contents'.
    exportReferencesGraph =
      let contentsList = if builtins.isList contents then contents else [ contents ];
      in map (x: [("closure-" + baseNameOf x) x]) contentsList;
    }
    ''
      printRegistration=1 perl ${pkgs.pathsFromGraph} closure-* > $out
    '';
  populateNixStore = args@{ contents, ... }:
    dockerTools.buildImage (args // {
      runAsRoot = dockerTools.shellScript "nix-db.sh" ''
        echo "Generating the nix database..."
        echo "Warning: only the database of the deepest Nix layer is loaded."
        echo "         If you want to use nix commands in the container, it would"
        echo "         be better to only have one layer that contains a nix store."
        # This requires Nix 1.12 or higher
        export NIX_REMOTE=local?root=$PWD
        ${nixUnstable}/bin/nix-store --load-db < ${nixRegistration contents}
      '';
    });
in
populateNixStore {
  name = "populated-example";
  contents = [ nix hello bash coreutils sqlite ];
}

Can we rewrite this PR to just add the above function to dockerTools?

[nlewo]

@puffnfresh yeah, really nice improvement. Thanks.
I then submitted a new patch with your implementation. I use extraCommand instead of runAsRoot to avoid the need of qemu. I hope that I don't miss something regarding runAsRoot.

[puffnfresh]

Even better. Looks good!

[nlewo]

@globin could you have a look?

[nlewo]

Maybe @grahamc you could have look?

[grahamc]

LGTM!

[nlewo]

Could we move forward on this PR ? :)

[nlewo]

Thanks

[nlewo]

@bjornfor I try to improve this in #29765