Consider/evaluate building FHS environments for containers

[tazjin]

This might lead to container images that people who are unfamiliar with Nix find more useful.

In practice, this would replace the symlink layer that is currently constructed via symlinkJoin in Nix with a layer that looks like an FHS layout. If there's a primitive in Nix for doing this, it might be preferable!

[wamserma]

Like buildFHSChrootEnv {} + chroot in the final layer?
(see also here http://sandervanderburg.blogspot.com/2013/09/composing-fhs-compatible-chroot.html)

[JohnAZoidberg]

I've done that, to run proprietary software in a container like:

let
  pkgs = import <nixpkgs> {};
  # Just build the FHS without the env part, that chroots you into it
  fhs = (pkgs.callPackage <nixpkgs/pkgs/build-support/build-fhs-userenv/env.nix> {}) {
    name = "container-fhs";
    targetPkgs = pkgs: with pkgs; [ hello cowsay ];
    multiPkgs = null;  # Don't include glibc's multilib
  };
in
  pkgs.dockerTools.buildImage {
  name = "custom-container";
  tag = "latest";

  contents = pkgs.symlinkJoin {
    name = "contents";
    paths = with pkgs; [
      fhs

      # Creating directories and files like this is much faster than runAsRoot
      (pkgs.runCommand "tempfiles" {} ''
        mkdir -p $out/{var/tmp,tmp}
      '')

      /more/paths
    ];
  };

  config = {
    Env = [
      "PATH=usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
      "LD_LIBRARY_PATH=/run/opengl-driver/lib:/run/opengl-driver-32/lib:/usr/lib:/usr/lib32:"
    ];
    Entrypoint = [ "${pkgs.bashInteractive}/bin/bash"];
  };
}

[tazjin]

@JohnAZoidberg Thanks, that's close to what I imagined - good to know that this already exists as isolated functionality!

The barebones FHS env still pulls in a lot of stuff that we don't want in container images - I'll take a look at how that can be reduced, and alternatively find a way to maybe gate the FHS feature (e.g. by introducing an fhs meta-package that triggers the behaviour).

[JohnAZoidberg]

Yeah, there's lots of room for tweaking in pkgs/build-support/build-fhs-userenv/env.nix.

Especially:

basePkgs = with pkgs;
  [ glibcLocales
    (if isMultiBuild then glibc_multi else glibc)
    (toString gcc.cc.lib) bashInteractive coreutils less shadow su
    gawk diffutils findutils gnused gnugrep
    gnutar gzip bzip2 xz
  ];

[tazjin]

@JohnAZoidberg Are you at NixCon by any chance? If so we could probably spend an hour or so hacking on this.

[JohnAZoidberg]

Yes, I am. Sure, that'd be fun! You can come to me after my talk. It's the first one tomorrow morning.