Skip to content

stackage2nix 1.0

Jump to bottom
Dmitry Bushev edited this page Jun 13, 2018 · 2 revisions

stackage2nix was started with a simple idea in mind to create a nix expression from a stack.yaml file without a clear understanding of how to do that. The first idea was to replicate the logic of stack itself. That's how we end up creating Stackage LTS in Nix first, and then overriding it with packages from stack.yaml. Then we noticed that the Stackage LTS is stable, it has all packages pinned to a specific revision; in opposite to Hackage, where one version of a package can have several revisions which have different dependency bounds and could be incompatible with other packages in LTS. So, since Stackage is stable, we can generate it only once. For that reason, we created typeable/nixpkgs-stackage overlay with Stackage snapshots.

During the ZuriHac I had an opportunity to talk to people who are actually using this thing, and I got the impression that the overlay was so far the most useful part because it doesn't have any dependencies and can be used as is. On the opposite side, the tool that creates the overlay is not easy to use because it requires several dependencies which are managed outside of Nix (see #41).

Separate stackage2nix and stack2nix

Now it becomes clear that stackage2nix logic should be split into two pieces. One executable that generates Stackage LTS, and the other executable that overrides given LTS with packages from stack.yaml.

The first part, stackage2nix would still have those clumsy unmanaged dependencies (until #41 is fixed), but it doesn't matter because end users use only the result of its work - the [packages-stackage] overlay.

The second part, codename stack2nix should generate an override of the Stackage LTS. This should only require a dependency to Cabal database, but it is already provided by nixpkgs as an all-cabal-hashes expression.

Goals

The primary goal of this refactoring is to get reproducible builds and better user experience.

Dependencies on commercialhaskell/all-cabal-hashes and Cabal database 01-index.tar.gz are the main issues when you want to use stackage2nix. So far they cannot be pinned to a particular revision. The work on #41 should help with that.

The most complicated thing in stackage2nix is generating a Stackage. We can reduce the number of dependencies and therefore build time of executable by splitting it onto stackage2nix and stack2nix. Two distinct executables should also eliminate the confusion about the purpose of each one. In addition to separation of the logic into two executables, it would be nice to:

  • (optional) make sure that stack2nix doesn't depend on stackage-curator because it brings the major part of a transitive dependencies
  • (optional) create static executables for both stackage2nix and stack2nix
  • (investigate) maybe it would be beneficial to replace the current NixOS/Cabal2Nix with the other (recently rewritten from scratch) tool angerman/Cabal2Nix
  • remove the dependency on hackage.haskell.org/01-index.tar.gz by creating it from the all-cabal-hashes repository

stackage2nix

The purpose is to generate the Stackage packages of provided revision.

Dependencies

TODO

The stackage2nix Nix wrapper should allow pinning all the required dependencies to a specific commit. That way user gets the control over the dependencies.

Remove the logic (CLI flags would be affected) related to stack2nix.

stack2nix (codename)

The purpose is to convert packages from stack.yaml to the corresponding Nix expressions that override Stackage LTS. Probably would be a good idea to come up with a different name to prevent the confusion with other similar tools.

Dependencies

Create a Nix wrapper that allows pinning commercialhaskell/all-cabal-hashes dependency to a specific commit.

TODO

The logic should be moved from the current stackage2nix

Ensure that it doesn't depend on the stackage-curator or/and create a static executable.

convert all-cabal-hashes to 01-index.tar

To eliminate dependency on 01-index.tar.gz we should be able to generate the index from commercialhaskell/all-cabal-hashes repository.

The tool hackage-repo-tool from haskell/hackage-security should do the trick. I was able only take a brief look into it, so proper investigation is required. Currently it could not be used as a library, so we need to have it on the runtime PATH in the Nix wrapper. Luckily nixpkgs provide haskellPackages.hackage-repo-tool. We can use one from nixpkgs or create own derivation with the callHackage helper.

$ nix-build '<nixpkgs>' -A haskellPackages.hackage-repo-tool --dry-run
these derivations will be built:
  /nix/store/fp8xlc5qanp4kz7xvsid1q865nmq16hv-hackage-repo-tool-0.1.1.drv
these paths will be fetched (4.70 MiB download, 59.44 MiB unpacked):
...

An additional source of information about the Cabal logic is the cabal-testsuite directory of the haskell/cabal repository.

Clone this wiki locally