Reproducibility of Haskell Stack development is broken on NixOS

[thomas-huegel]

Describe the bug

The command stack new suddenly breaks without any configuration change. This may be because the new version expects a new GHC version which is absent from the current distribution.

Steps To Reproduce

It happens because the Stack resolver seems to be downloaded from the Web. So it gets updated independently from the NixOS packages.

Expected behavior

Nothing should be updated automatically by default.
Stack should be pinned to a version compatible with the current NixOS release.
If stack new works on Monday on some machine, it should work on Tuesday on the same (not updated) machine.

[cdepillabout]

@thomas-huegel I imagine the problem you're seeing is that by default stack uses your current NIX_PATH (and your current channel) to automatically download GHC. But it fails if stack can't find the GHC it is looking for in your current channel.

Here's an example of what this looks like:

$ stack --nix new --bare my-haskell-example
Using latest snapshot resolver: lts-20.2
error: attribute 'ghc925' missing

       at «string»:1:43:

            1| with (import <nixpkgs> {}); let inputs = [haskell.compiler.ghc925 git gcc gmp]; libPath = lib.makeLibraryPath inputs; stackExtraArgs = lib.concatMap (pkg: [ ''--extra-lib-dirs=${lib.getLib pkg}/lib''   ''--extra-include-dirs=${lib.getDev pkg}/include'' ]) inputs; in runCommand ''myEnv'' { buildInputs = lib.optional stdenv.isLinux glibcLocales ++ inputs; STACK_PLATFORM_VARIANT=''nix''; STACK_IN_NIX_SHELL=1; LD_LIBRARY_PATH = libPath;STACK_IN_NIX_EXTRA_ARGS = stackExtraArgs; LANG="en_US.UTF-8";} ""
             |                                           ^
       Did you mean one of ghc923 or ghc902?

You can see that Stack is trying to use the lts-20.2 resolver, which uses GHC-9.2.5. But my current Nix channel doesn't contain haskell.compiler.ghc925, which can be confirmed like the following:

$ nix repl '<nixpkgs>'
nix-repl> haskell.compiler.ghc925
error: attribute 'ghc925' missing

       at «string»:1:1:

            1| haskell.compiler.ghc925
             | ^
       Did you mean one of ghc923 or ghc902?

When I explicitly use a newer channel (nixpkgs-unstable from just two days ago), it works, because this channel contains GHC-9.2.5:

$ nix repl -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/be9e3762e719211368d186f547f847737baad720.tar.gz '<nixpkgs>'
nix-repl> haskell.compiler.ghc925
«derivation /nix/store/92q3dy0ccj9k5lpsl3cxqzbzvdl5daqx-ghc-9.2.5.drv»

With stack:

$ stack --nix --nix-path nixpkgs=https://github.com/NixOS/nixpkgs/archive/be9e3762e719211368d186f547f847737baad720.tar.gz new --bare my-haskell-example
these 2 paths will be fetched (160.33 MiB download, 2245.14 MiB unpacked):
  /nix/store/1bj5rzwny8qxyy5rlvpahw1pb2g8ihlm-ghc-9.2.5-doc
  /nix/store/625ha5v4kdpz8z8py8z57j7xy7dfd0dr-ghc-9.2.5
copying path '/nix/store/1bj5rzwny8qxyy5rlvpahw1pb2g8ihlm-ghc-9.2.5-doc' from 'https://cache.nixos.org'...
...

You could alternatively use an older Stackage resolver that uses a GHC version that is in your current Nix channel:

$ stack --nix new --bare my-haskell-example --resolver lts-19.33 # this is GHC-9.0.2, which is in my current channel
these 2 paths will be fetched (160.33 MiB download, 2245.14 MiB unpacked):
  /nix/store/1bj5rzwny8qxyy5rlvpahw1fb3f8ihlm-ghc-9.0.2-doc
  /nix/store/625ha5v4kdpz8z8py8z57j7xy7dfd0dr-ghc-9.0.2
...

---

My understanding is that stack's functionality for downloading a GHC to your system is certainly helpful, but it is not meant to be 100% reproducible in all cases. In fact, it is quite common for people to run into this problem:

• https://discourse.nixos.org/t/stack-new-error-attribute-ghc-missing/23405
• https://discourse.nixos.org/t/haskells-stack-failing-trouble-understanding-package-system/18320
• https://discourse.nixos.org/t/can-i-use-haskells-stack-with-nix-run-for-a-resolver-not-in-my-current-nixpkgs/17317
• https://discourse.nixos.org/t/haskell-stack-not-working-after-upgrade-to-19-03/2687

There are two general ways to make working with stack more reproducible:

1. Create a stack-shell.nix file and pin Nixpkgs and GHC in that file. Two examples:
   • https://github.com/cdepillabout/termonad/blob/fcbcf918eb44871d5d7ecc33f191d01cade7078d/stack.yaml#L40 and https://github.com/cdepillabout/termonad/blob/fcbcf918eb44871d5d7ecc33f191d01cade7078d/.nix-helpers/stack-shell.nix
   • https://github.com/cdepillabout/stacklock2nix/blob/8b32b2aaf0c9d9a7c342be818b4e941b09067bbf/my-example-haskell-lib-advanced/stack.yaml#L76 and https://github.com/cdepillabout/stacklock2nix/blob/8b32b2aaf0c9d9a7c342be818b4e941b09067bbf/my-example-haskell-lib-advanced/nix/stack-shell.nix
     The benefit to this method is that you can continue using stack like normal, and mostly ignore all the Nix-related parts. stack re-execs itself in a nix-shell behind the scenes, so you don't explicitly have to run nix-shell yourself.
2. Create a full Nix development environment, and force stack to use the system GHC: https://docs.haskellstack.org/en/stable/nix_integration/#supporting-both-nix-and-non-nix-developers

[cdepillabout]

I don't think that it is really a Nixpkgs problem that Stack isn't fully reproducible out-of-the-box (without pinning anything). There are lots of examples of other CLI tools that Nixpkgs provides that aren't fully reproducible, and it is generally not considered a Nixpkgs problem. (I guess Stack is a little unique, since it does have Nix integration, unlike a lot of other tools.)

I'm going to go ahead and close this, but if you feel strongly about this, maybe you could create an issue on the upstream Stack repo? Or you could try to argue why the way Stack works by default is actually a Nixpkgs problem (and not a Stack problem).