Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stack2nix? #212

Closed
proger opened this Issue Nov 13, 2015 · 22 comments

Comments

Projects
None yet
6 participants
@proger
Copy link

proger commented Nov 13, 2015

I was trying to use nixpkgs to build a stack-based program and ended up with an ugly hack that selectively parses stack.yaml and runs cabal2nix a couple of times. Do you think cabal2nix should support stack files?

@peti peti added the enhancement label Nov 13, 2015

@peti

This comment has been minimized.

Copy link
Member

peti commented Nov 13, 2015

It sure would be nice if it did.

@bennofs

This comment has been minimized.

Copy link
Member

bennofs commented Jan 21, 2017

I plan to work on this, including support for generating the correct stackage snapshot package set on the fly.

@domenkozar

This comment has been minimized.

Copy link
Member

domenkozar commented Jan 21, 2017

@bennofs let's collaborate this is also on my todo

@bennofs

This comment has been minimized.

Copy link
Member

bennofs commented Jan 21, 2017

@domenkozar WIP implemention of stackage2nix at https://github.com/bennofs/cabal2nix/tree/feature-stackage2nix. Successfully generates a haskell-packages.nix for latest stackage LTS, but I need to do some more testing before I can submit that as a proper PR

@bennofs

This comment has been minimized.

Copy link
Member

bennofs commented Jan 21, 2017

For stack2nix, I would use that to first generate a packages.nix for the snapshot specified in the stack file and then generate the necessary overrides for stack's extra packages

@domenkozar

This comment has been minimized.

Copy link
Member

domenkozar commented Jan 22, 2017

IIRC cabal2nix used to have stackage2nix.

@bennofs

This comment has been minimized.

Copy link
Member

bennofs commented Jan 22, 2017

@domenkozar yeah but that was a hack using some sed magic. I want an exact stackage2nix so it 100% matches what stack would build (modulo nix-specific configure options to help find libraries, but that should be everything that is different).

We also get flags etc from the snapshot .yaml, the previous approach just used the cabal constraints for the versions of a stackage snapshot.

@peti

This comment has been minimized.

Copy link
Member

peti commented Jan 22, 2017

Initially, we did take the Cabal flags from Stackage, but ultimately dropped that feature again and relied on auto-detection henceforth. Keep in mind that Stackage builds only on Linux/x86_64 running Ubuntu. Nixpkgs, however, may run on 32 bit Linux as well as Darwin etc., so the flags given by Stackage sometimes don't fit. On some occasions, the flags fit but we don't want them anyway, i.e. in case of git-annex. Stackage can't buid the full version because some dependencies aren't in Stackage. We can (and want to) build the full version, however.

@bennofs

This comment has been minimized.

Copy link
Member

bennofs commented Jan 22, 2017

@peti well I understand that this makes sense for hackage-packages.nix, but if you want to use the snapshot to build packages with a stack.yaml, then those packages probably expect git-annex not to be the full version (because they build against Stackage with stack)

@bennofs

This comment has been minimized.

Copy link
Member

bennofs commented Jan 22, 2017

@peti I was under the impression that stack also used these flags when it builds against a stackage snapshot. So apparently, it doesn't do that?

@peti

This comment has been minimized.

Copy link
Member

peti commented Jan 22, 2017

I haven't actually investigated this issue, so I can't say for sure. My impression was that the flags recorded in the build plan are simply whatever Cabal auto-detected during the test builds on Travis. I may be wrong though.

@domenkozar

This comment has been minimized.

Copy link
Member

domenkozar commented Jan 23, 2017

@bennofs did you check how hard would it be to ask stack for the final build plan and then get a list of packages with versions? Using that we could generate the package set.

I think it's worth investigating more at which stage we hook in as that brings lots of consequences on stability.

@bennofs

This comment has been minimized.

Copy link
Member

bennofs commented Jan 23, 2017

@domenkozar that's a good idea. But I would have to look into stack's source code to understand how exactly stack does build plan construction internally and if retains enough information for us.

@domenkozar

This comment has been minimized.

Copy link
Member

domenkozar commented Jan 23, 2017

I think it's worth seeing how easy is that.

@bennofs

This comment has been minimized.

Copy link
Member

bennofs commented Jan 23, 2017

@domenkozar I think I dealbreaker here is how stack handles git dependencies: it'll check them out before resolving dependencies, which is not viable for generating nix expressions (the generation should be free of network accesses, so it works in import-from-derivation as well. Any git-dependencies should be fetched using fetchgit)

@puffnfresh

This comment has been minimized.

Copy link
Contributor

puffnfresh commented Jan 25, 2017

I started doing something like this a while ago, I got some information from Stack being put into derivations:

https://github.com/puffnfresh/stack/tree/feature/full-nix

@domenkozar

This comment has been minimized.

Copy link
Member

domenkozar commented Feb 14, 2017

So we're talking about two use cases here:

  1. stack --full-nix which would actually use nix for building and could use binary cache

  2. Generating Nix expressions to be used outside stack based on stack.yaml

Both are really useful! @puffnfresh how far did you get on 1)?

@4e6

This comment has been minimized.

Copy link
Contributor

4e6 commented Feb 20, 2017

@bennofs stackage2nix looks good. I tested it with the following nixpkgs fork:
4e6/nixpkgs@release-16.09...4e6:stackage
where I defined lts-6.25 compiler and was able build our project with it.

The only issue I had with stackage2nix is that compiler config should have package names unquoted to be able to override them, see commit Fix: unquote assignments

@4e6

This comment has been minimized.

Copy link
Contributor

4e6 commented Jun 26, 2017

A small update. I continued @bennofs work on stackage2nix and made a stack2nix tool on top of it. It takes stack.yaml and creates a Nix derivation with all Stackage packages plus overrides from stack file.
Also, there is a stack2nix-examples repo that configured to build several public projects with stack2nix. I used this repo as a test to verify the correctness of result derivations but then decided to publish it as well, as it is a good source of examples.

@domenkozar

This comment has been minimized.

Copy link
Member

domenkozar commented Jun 27, 2017

We have also made stack2nix at https://github.com/input-output-hk/stack2nix

It takes a bit different approach, only including packages that are part of stack list-dependencies.

@domenkozar

This comment has been minimized.

Copy link
Member

domenkozar commented Sep 2, 2017

There are two packages, each taking a different approach, but implementing a basic stack2nix:

@peti perhaps we can close this issue as "out of scope" for cabal2nix?

@peti

This comment has been minimized.

Copy link
Member

peti commented Sep 2, 2017

OK.

@peti peti closed this Sep 2, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.