Enable Minimal Platform as Default via a Platform Collection

[gbaz]

This is the thread to discuss how to implement one section of the "Improving the Get Haskell Experience" proposal -- namely enabling the installer to be "minimal" and only include core libraries and tools rather than all platform libraries.

The issue that arises in doing so is then how to provide some notion of the existence of this blessed set that is visible and usable from cabal as well as stack.

On the server side, approaches include collections (https://mail.haskell.org/pipermail/libraries/2015-July/025954.html) and just adding the platform as a special set to stackage.

On the client side, approaches include again "collections" or alternately a sort of shell-script to swap between an "additional layer" of package dbs that serves at the resolver level.

In this thread I hope we can discuss the most straightforward and rapid to implement approach towards this goal, bearing in mind that we may also produce better more general solutions later.

(Please note that this thread is for technical discussion of how to achieve this goal, and broader questions of "why" or "why not" or "what about other goals" should be held elsewhere 😄 )

[snoyberg]

When this came up at ICFP in 2014, I recommended just using stackage.org for the server side functionality, since it's already built for providing that. Something even simpler is possible: a static text file hosted anywhere (I'd recommend Github repo and use the raw links). I do not believe we should block this on changes to Hackage server, which is exactly what happened last time around.

For the client side, I think we need to clarify what the goal is. I can see multiple different ideas here:

1. Be like Stackage today: specify exact versions of specific packages, and allow any other version of other packages.
2. Be like the old "Stackage exclusive" (no longer recommended): require this set of packages at a specific version, and no other packages.
3. Something else: is this just an advisory set of packages, but cabal will be free to choose other versions at will?

Related question: does the user have the right/ability to override the restrictions given above?

I can explain how stack handles all of these, which IMO is the best way forward. However, one of the fundamental advantages we saw with stack vs the old cabal.config approach is that we could express things that we couldn't do with cabal. Maybe there are other approaches I haven't explored.

[gbaz]

I agree that changes to the hackage server shouldn't be a blocker here, but just a future nice-to-have.

I'm fine with 1, and yes certainly users should be able to "opt-out" or switch resolvers so to speak, etc.

The issue I see here is that the cabal.config approach needs to be done package-by-package at the moment. Perhaps there is a minimal patch to cabal to enable it to be done default-globally? (I know there are other ideas kicking around longer term for cabal, involving more flexible global things, etc. But I would want this more to be just a short-term "minimal" plan.)

[snoyberg]

It can certainly be done globally, e.g. global config instructions for LTS-3.0. We'd just need to patch cabal to allow us to add arbitrary text to be included in the default generated config file.

[gbaz]

Patching things directly into the global config is hard to "swap out" easily.

Maybe we can let the global config point to an additional "extra-constraints" file? That sounds like a very quick patch.

Then by swapping what file it points to, people can pick platform, lts, etc. with minimal hassle.

[gbaz]

I like this idea, but I'd be interested to see if @mzero @dcoutts @mboes and others have considerations here that we're overlooking.

[snoyberg]

How will the platform get these cabal changes? I see three options:

1. Release an unofficial cabal binary
2. Ask the cabal team to release a point fix on the current release with this feature
3. Wait until the next official cabal release

[gbaz]

number 2. Would be the plan. Assuming we can get agreement on the patch and get it done, I'm sure we could negotiate a point fix for this feature.

[snoyberg]

I think Github ate your text as markdown ;) Did you mean (2)?

[gbaz]

yes, edited it to circumvent github's cleverness.

[snoyberg]

Can we define the overarching goal of the feature we're trying to add to the platform here? Is the goal to avoid the cabal dependency solver being able to choose incompatible versions of platform packages? Or avoid upgrading to newer versions that haven't been blessed? Or something else? I know you started off saying we'd avoid the "why," but the "why" would probably help us (or at least me) understand the various trade-offs.

[mboes]

Here's a proposal: stick to the plan proposed by @mzero and @snoyberg in a recent email. Namely, don't install anything in the global db anymore, include stack in HP, but ship a new version of the HP as soon as practical so as to resolve the current ambiguity of "HP or min installers or stack or ..." for new users quickly. I'm not sure that this involves much work at all. I'm surely missing some details, but as @gbaz says, the point is to make the smallest and most straightforward increments forward that will allow us to provide a unified view to new users.

The increments I see necessary are:

• Strip the HP down to only including binaries (ghc, cabal, stack, alex, ...).
• Add a new set of haskell-platform-* resolvers to stack corresponding to HP releases, which uniquely identify a set of HP libraries and their versions.
• Provide a cabal.config to cabal users, which effectively does exactly same thing as a resolver does for stack. Ideally this should be settable globally for those users that wish to do that.

What we don't need to wait on, I think, is cabal and/or Hackage support for "collections". When/if that feature materializes, the stack resolvers above could map to a collection, or not - either way it shouldn't matter to users.

[gbaz]

The goal, in my mind at least, is to provide a system whereby you don't get package plans incompatible with the platform as a fixed-at-release set of versions, and to enable this from cabal as well as stack.

The current way this is enforced -- by sticking things in the global db -- is obviously bad and should be changed. But the fact that we can do this in some way is good, and should not be abandoned :-)

[mboes]

Hm, laggy response apparently. So the tl;dr of my last comment is basically: I agree with what was said so far.

I think we should have as a sole goal that those cabal users that want to develop HP compatible packages can easily check compatibility. The corresponding problem for stack is essentially solved once HP resolvers are added. Avoiding recompiles of HP libraries would be a "nice to have" in my book, not a blocker for converging on a new release.

[gbaz]

I have a repo with a version of the patch I was discussing (globally settable cabal.config) here: https://github.com/gbaz/cabal/tree/constraints-config and kicked off a discussion cabal-devel about how to polish it and get it in.

As a nice bonus, this should make using stackage from cabal more pleasant in general as well, since users can just set their global cabal.config to a stackage lts or the like instead of going project by project.