Can't nest ghcWithPackages environments

[Warbo]

We can use haskellPackages.ghcWithPackages to get an instance of GHC with particular packages available, e.g.

~$ nix-shell -p 'haskellPackages.ghcWithPackages (h: [ h.text ])'
[nix-shell:~]$ echo -e 'import Data.Text\nmain = putStrLn "hello world"' | runhaskell
hello world

However, if we invoke ghcWithPackages from within that shell/environment, we get the wrong packages available, e.g.

~$ nix-shell -p 'haskellPackages.ghcWithPackages (h: [ h.containers ])'
[nix-shell:~]$ nix-shell -p 'haskellPackages.ghcWithPackages (h: [ h.text ])'
\[\][nix-shell:~]$\[\] echo -e 'import Data.Text\nmain = putStrLn "hello world"' | runhaskell

/run/user/1000/runghcXXXX1804289383846930886.hs:1:8:
    Could not find module ‘Data.Text’
    Perhaps you meant
      Data.Set (from containers-0.5.6.2@conta_2C3ZI8RgPO2LBMidXKTvIU)
    Use -v to see a list of the files searched for.

This is because the outer GHC appears first in PATH:

~$ nix-shell -p 'haskellPackages.ghcWithPackages (h: [ h.containers ])'' which
[nix-shell:~]$ which runhaskell
/nix/store/x60g7pf7d6273b593p74nra6wq5xviad-ghc-7.10.3/bin/runhaskell

[nix-shell:~]$ nix-shell -p 'haskellPackages.ghcWithPackages (h: [ h.text ])'
\[\][nix-shell:~]$\[\] which runhaskell
/nix/store/x60g7pf7d6273b593p74nra6wq5xviad-ghc-7.10.3/bin/runhaskell

\[\][nix-shell:~]$\[\] echo "$PATH" | tr ':' '\n' | grep ghc
/nix/store/x60g7pf7d6273b593p74nra6wq5xviad-ghc-7.10.3/bin
/nix/store/9b7zi2dbnnl38y56smgfqh2ma6h39ghx-ghc-7.10.3/bin
/nix/store/x60g7pf7d6273b593p74nra6wq5xviad-ghc-7.10.3/bin

We can avoid this by using --pure on the inner nix-shell, but that may prevent some functionality from working; in particular, we can no longer call nix-shell; either because it's not available, or if we specified it as a dependency we get an error:

$ nix-shell --pure -p nix
[nix-shell:~]$ nix-shell -p which
error: Nix database directory ‘/nix/var/nix/db’ is not writable: Permission denied

This problem with the environment not appearing at the head of PATH seem to be the cause of some errors I've been running into with some Haskell applications. Hopefully those more familiar with Nix, buildEnv, etc. could figure out a way to ensure this directly.

In the mean time, I've figured out a hacky way to fix the PATH after the fact. First we distinguish between the environments by name, rather than by hash; ghcWithPackages inherits its name from the relevant ghc package, which is why both environments above were called xxxx-ghc-7.10.3.

After stumbling on the comments for #6575 I tried using lib.setName to set their names, as per #6575 (comment) but this doesn't work; e.g.

~$ nix-shell -p 'lib.setName "ghc-env-with-containers" (haskellPackages.ghcWithPackages (h: [ h.containers ]))'
[nix-shell:~/]$ nix-shell -p 'lib.setName "ghc-env-with-text" (haskellPackages.ghcWithPackages (h: [ h.text ]))'
\[\][nix-shell:~]$\[\] echo "$PATH" | tr ':' '\n' | grep ghc
/nix/store/x60g7pf7d6273b593p74nra6wq5xviad-ghc-7.10.3/bin
/nix/store/9b7zi2dbnnl38y56smgfqh2ma6h39ghx-ghc-7.10.3/bin
/nix/store/x60g7pf7d6273b593p74nra6wq5xviad-ghc-7.10.3/bin

Next I tried using overrideDerivation as per #6575 (comment) which seems to work, but causes GHC to rebuild from source, which I didn't want:

$ nix-shell -p '(haskellPackages.ghcWithPackages (h: [ h.containers ])).overrideDerivation (d: { name = "ghc-env-with-containers"; })'
these derivations will be built:
  /nix/store/aigi6g7xg1pcbs9xgc8fqgf2as99m8i8-ghc-env-with-containers.drv
building path(s) ‘/nix/store/kv5bh99yck9zlyn68wsg8ng4wy1gcgdr-ghc-env-with-containers’
unpacking sources
unpacking source archive /nix/store/fwiy0ninbnwmijp6njngyqkjam6c0axh-ghc-7.10.3-src.tar.xz
source root is ghc-7.10.3
setting SOURCE_DATE_EPOCH to timestamp 1448928600 of file ghc-7.10.3/utils/hpc/HpcParser.hs
patching sources
applying patch /nix/store/f49wn9xld3cpf7q45f0apjnbq53q1vyr-ghc-7.10.3a.patch
patching file ANNOUNCE
...

My current solution is to use buildEnv:

$ nix-shell -p 'buildEnv { name = "ghc-env-with-containers"; paths = [(haskellPackages.ghcWithPackages (h: [ h.containers ]))]; }'
[nix-shell:~]$ nix-shell -p 'buildEnv { name = "ghc-env-with-text"; paths = [(haskellPackages.ghcWithPackages (h: [ h.text ]))]; }'
\[\][nix-shell:~]$\[\] echo "$PATH" | tr ':' '\n' | grep ghc
/nix/store/daygyb4lcdbkw4qjqb1d3ls4a9jxdjkh-ghc-env-with-containers/bin
/nix/store/zxchs1zbc9dandr6wxylmbk7h5panw32-ghc-env-with-text/bin
/nix/store/daygyb4lcdbkw4qjqb1d3ls4a9jxdjkh-ghc-env-with-containers/bin

The next step is to look up these names in PATH and prepend the correct one to the front, which is easy enough if the name of the environment is in $NAME:

DIR=$(echo "$PATH" | grep -o -- "[^:]*${NAME}/[^:]*")
export PATH=$DIR:$PATH

With this workaround, I've able to create nested Haskell environments where each "layer" has access to the packages it needs.

I admit that it's rather an obscure use-case, so my main reason to report it is to help anyone encountering such issues in the future. If it doesn't seem worth fixing then I don't mind continuing to use my workaround.

For a little context, I was testing an application which uses nix-shell internally to resolve Haskell dependencies, since they're only known at run-time. It runs fine standalone, but trying to invoke it as a sub-process from other Haskell programs (e.g. for tests or benchmarks) runs into this problem.

I'm running NixOS from the nixos-unstable channel:

$ nixos-version
16.03pre75806.77f8f35 (Emu)

[peti]

A related discussion is taking place on the nix-dev mailing list: http://lists.science.uu.nl/pipermail/nix-dev/2016-February/019531.html.

[peti]

It appears that nesting of nix-shell environments is not supported.

[Warbo]

Yes, I agree this should be closed.

For the record, it is possible to work around this, by using pure shells and setting NIX_PATH and NIX_REMOTE appropriately. Whether that's a good idea or not is another question ;)