Please support Stackage LTS 8.x

[peti]

The current release of stack, version 1.3.2, cannot be compiled in an LTS-8.x-based environment because of the following restrictions: directory >=1.2.1.0 && <1.3, hpack >=0.14.0 && <0.16. Both version constraints can simply be relaxed to include the respective new versions; the build succeeds fine with the newer dependencies.

[decentral1se]

@peti, those bounds are already relaxed, from looking in the stack.cabal in HEAD.

Is getting Stack from source, or running a stack upgrade --git acceptable for you?

[sjakobi]

One problem with compiling stack with GHC-8.0.2 is mgsloan/store#91.

It's probably necessary to release a fixed version of store before stack can be compiled with reasonable memory requirements.

[decentral1se]

Ah yes, I did see an OOM on https://travis-ci.org/commercialhaskell/stack/jobs/200976309 for #3003.

[borsboom]

I've updated the hackage metadata to relax these (and a few other) upper bounds.

[decentral1se]

FYI: New store is out and #3009 covers it.

[peti]

Why has this issue closed? Stack still doesn't build in the context of LTS-8.x because it doesn't allow compilation against store-0.4.*, which is forced due to store-core-0.4 being part of the LTS-8 package set. Am I missing something?

[decentral1se]

Stack does build for LTS-8.x, have you got the latest version?

[peti]

stack-1.4.0 depends on store >=0.2.1.0 && <0.4, which depends on store-core ==0.3.*, which conflicts with conflicts with store-core == 0.4 from LTS-8.5.

[borsboom]

Note https://github.com/commercialhaskell/stack/blob/v1.4.0/stack.cabal#L276 has no upper bound:

, store >= 0.2.1.0

Maybe you're looking at the dependencies on Hackage, which have stricter bounds because we use stack upload --pvp-bounds=both to try to ensure that people who install Stack using cabal-install use similar dependency versions as we do for official builds, but if you're using Stack you should be building on LTS 8 using https://github.com/commercialhaskell/stack/blob/v1.4.0/stack-8.0.yaml.

Either way, I'll relax the upper bound for store on Hackage.

[alanz]

@borsboom It looks like the metadata change broke the installplan on hackage. It worked fine with the prior setting.

See http://104.239.175.197:8080/package/stack#GHC-8.0/stack-1.4.0
(this is a test version of the new matrix.hackage.org, still under development, may change at any time)

The log for the failure is at the bottom of the page

[peti]

Maybe you're looking at the dependencies on Hackage.

Yes, I did.

Either way, I'll relax the upper bound for store on Hackage.

Very cool, thank you!

[alanz]

wrt my prior comment, if necessary I can try to dig in and find the issue/resolution.

[borsboom]

@alanz: I've reverted the metadata change. I should have tested more carefully before making it.

@peti: It turns out that LTS 8.0 didn't include store-core at all, so that always had to be added as an extra-dep in stack.yaml. store-core-0.4 was was added in LTS 8.5 (occasionally new packages are added to existing LTS major versions), but stack-1.4.0 still needs the same extra-dep to use store-core-0.3. Compatibility with store-core-0.4 was added in d363712, which will be included in the next release.

[alanz]

Thanks. The benefits of having additional sanity checking out there.

[alanz]

I can confirm that it is green again at http://104.239.175.197:8080/package/stack

[peti]

http://hackage.haskell.org/package/stack-1.4.0 cannot be built within LTS-8.x because it depends on old versions of cryptonite.

[mgsloan]

@peti Hmm, that's surprising! The config for building 1.4.0 with ghc 8.0 uses lts-8.0's version of cryptonite, 0.21. Cryptonite's changelog doesn't seem to indicate anything breaking backwards compat in 0.22 - https://github.com/haskell-crypto/cryptonite/blob/master/CHANGELOG.md .

HEAD version of stack is using lts-8.5 and doesn't specify cryptonite as an extra dep.

[peti]

http://hackage.haskell.org/package/stack-1.4.0 requires cryptonite ==0.19.*, so it cannot build in LTS-8.0, which has cryptonite-0.21.

[mgsloan]

Oh I see, it's because of using --pvp-bounds when uploading the package, argh!

[borsboom]

#2262 would be really handy for avoiding this sort of thing. In the mean time, I'll update the release process to check for any bounds like this and add explicit upper bounds to the stack.cabal.

[borsboom]

Actually other option would be to stop using --pvp-bounds entirely. I wonder if I should try that and just see what breaks :)

[borsboom]

Alright, I think all the bounds should be adjusted now, and I have a documented process to follow in the future. In addition to cryptonite, I bumped cryptonite-conduit, memory, and template-haskell.

[alanz]

@borsboom There is still a problem on hackage: http://104.239.175.197:8080/package/stack#GHC-8.0/stack-1.4.0

@peti, How can I reproduce the problem nix has, if I have nixpkgs set up under debian? I want to see if I can come up with something that

• builds with lts-6.25 (which is constrained to store-0.3)
• builds with lts-8.x (store-0.4.*)
• builds with nix
• has a solution on hackage for GHC 7.10.3 and GHC 8.0.2

I suspect that limiting store based on GHC version might do the trick.

[alanz]

From https://docs.haskellstack.org/en/stable/faq/#i-need-to-use-a-different-version-of-a-package-than-what-is-provided-by-the-lts-haskell-snapshot-im-using-what-should-i-do it seems that store-0.3 will always be used?

As per

stack/stack.yaml

Line 17 in e714f1d

- store-0.3

[peti]

@alanz, Nix has no trouble building stack. These slightly weird version constraints are inconvenient for Nixpkgs maintainers because we have to add config know-how into the package database manually (instead of being able to "just build it" within the default LTS 8.x package set), but at least it's possible to do that. Problems arise on other distributions (like openSUSE), which cannot easily have multiple versions of the same package at the same time (like Nix can). There, adding another version of store or cryptonite that's used only in the stack build but not anywhere else is very difficult and very inconvenient.

[alanz]

Thanks. So then the root problem from a hackage point of view is that the constraint on store should be < 0.4, which will not inconvenience anyone else.

[peti]

store < 0.4 would certainly inconvenience everyone who's basing their package set on LTS-8.x since that standard mandates store-core 0.4.*.

[alanz]

Except that the stack.yaml for stack has

...
extra-deps:
- Cabal-1.24.2.0
- th-utilities-0.2.0.1
- store-0.3
...

And I understand that this means that although store-0.4 is in the lts, for building stack it will use store-0.3.

[peti]

No distribution I am aware of uses stack to build Haskell packages, because stack builds are inherently non-deterministic due to network access, etc. The way builds are run in those systems I am aware of is to execute runhaskell Setup.hs build etc. in an environment that's been set up by the distribution's build manager, and usually the choice of packages available for these environments cannot contain multiple versions of the same package. Therefore, systems that follow LTS 8.x cannot provide store < 4.

[mgsloan]

stack builds are inherently non-deterministic due to network access

I don't believe that this is the case, it only fetches things that are stable (particular ghc version, particular stackage snapshot). Hackage index updates don't affect package selection.

[peti]

Hackage index updates don't affect package selection.

Thanks to the wonders of destructive edits on Hackage, they do. A build that succeeds now very well fail after an Hackage update (and vice versa). Anyway, there are many reasons why stack is not ideal for driving reproducible distribution builds; the necessity for network access is just one of several issues.

[peti]

Dudes, is there any intention from your side to make stack compile in an LTS-8-based environment? Or can I just give up trying to distribute this tool and stop worrying?

[borsboom]

The currently released Stack version needs store-0.3, but the master branch has already upgraded to store-0.4 and should only need lts-8.11 packages, so the next version should be fully LTS-8 compatible.

That said, no guarantee it will stay that way for for all versions. Stack is an application not a library so we don't put a lot of effort to keep it compatible with a wide range of Haskell library versions since it doesn't need to "coexist" with them (it can just be installed from its own sandbox). We do make sure it can be built using the the latest couple of stable major GHC versions, though.