Hackage changes render cabal.config settings inconsistent and result in configure failure

[gracjan]

Story:

1. I used cabal freeze to have cabal.config in place.
2. Things were running smoothly for a couple of months.
3. Recently did 'cabal update'.
4. cabal install --only-dep stopped working with 'Dependency tree exhaustively searched'.
5. Applying a change to cabal.config solves problem:

   •         transformers-compat ==0.3.3.4,
     

   •         transformers-compat ==0.3.3.3,
     

6. Works again.

Now the real question:

How do I know why this suddenly stopped resolving? Did somebody change version ranges on some of dependent packages? How do i know which one got changed?

[23Skidoo]

Did somebody change version ranges on some of dependent packages?

Yes, probably. Guess that these kinds of errors are an unfortunate consequence of allowing to edit .cabal files on Hackage post-release.

[23Skidoo]

/cc @benarmston

Maybe cabal freeze could remember X-Hackage-Revision of dependent packages and print a warning when that changes?

[cartazio]

@23Skidoo making cabal freeze smarter in that way seems like a good solution.

[dcoutts]

Given that the source code cannot change, if freeze finds a solution and it builds then that's ok, and anyone who makes the constraints on hackage tighter and so makes a working solution break is probably doing something wrong. So perhaps cabal freeze should also make use of a feature to say that once it's picked those versions then we can actually ignore the constraints on those packages, and then we'd not be affected by mistaken constraint tightening on hackage.

[dcoutts]

That said, we may well want to still warn in such a situation. Suppose for sake of argument that in the original report, the solution included foo-1.0 which depended on transformers-compat, and that someone found a critical bug in foo-1.0 triggered by using transformers-compat-0.3.3.4 but that worked fine with earlier versions, so they added a constraint. Now my frozen package did appear to work, but I may well like to know that foo actually had a good reason to want the earlier transformers-compat so I can decide if I want to change or not.

So, if we follow my suggestion above of ignoring the constraints on the frozen versions, what should we warn about? If any packages within the solution now have constraints that conflict with the frozen version.

Actually it's worse than this in the case of a partially frozen solution. Suppose I freeze just some packages in my solution, I'd only want to ignore constraints on that package from existing things, and not ignore it for new things. I think to be on the safe side, it only makes sense to ignore the constraints on frozen packages if you've frozen everything, and not to do it for a partial freeze.

[mietek]

This seems like layering even more complexity on an already quite complex system. Can we simply disallow editing .cabal files post-release?

[cartazio]

No.
On Nov 20, 2014 5:27 AM, "Miëtek Bak" notifications@github.com wrote:

This seems like layering even more complexity on an already quite complex
system. Can we simply disallow editing .cabal files post-release?

—
Reply to this email directly or view it on GitHub
#2222 (comment).

[lpsmith]

Would it be sensible to ignore dependency ranges once you've frozen the dependency versions? Or perhaps it would be better to issue a warning when the cabal.config file specifies a manifest that's inconsistent with the dependency ranges. In any case, it seems nonsensical to issue an error.

[cartazio]

one thing that could be done is have freeze log the revision number of each package, and the have cabal issue a warning when the revision number doesn't match the one on hackage? (in some sense, we want a hackage snapshotting consistentcy check? because freeze is kinda saying that things built with a fixed snapshot of hackage right?)

[lpsmith]

I'm not sure I agree... I mean, once you have a frozen manifest of package versions, the version ranges in the cabal file are pretty much immaterial.

Of course, some things in the cabal file may not be immaterial, such as compilation flags. But I'm not sure I agree with changing those parts of a .cabal file without releasing another version... in my mind, the only form of mutability I ever really supported was version ranges. (And possibly things like synopsis/description, as those should be immaterial on the build itself.)

[mietek]

@cartazio: For this idea to work, each revision of each Cabal package description would need to be permanently available on Hackage, and not just disappear into the æther, as they currently do. Furthermore, cabal-install would need to be able to reference any package not only by its version, but also by its revision, in every single command and sub-command.

Given the number of unresolved issues with the existing functionality of cabal-install, I am strongly opposed to introducing Cabal package description revision numbers at the cabal-install level, for what is currently a Hackage-specific feature.

[lpsmith]

I suppose in the end, I'm not in flat-out disagreement with your suggestion that cabal freeze tracks the version of the cabal file used as well; there shouldn't really be any difference between ignoring the dependency ranges and freezing the cabal file if the only thing that changes is the dependency constraints.

The idea of building against a snapshot of hackage, on the other hand, I do disagree with on a couple of points. Being able to conveniently work with hackage snapshots may be useful for other reasons, but it seems to me to be a very strange, difficult, and possibly inconvenient way to not quite achieve repeatable builds: for starters, this still doesn't freeze the particular constraint solver you are using.

And, once you have a manifest of nailed-down versions, the constraint solver is not really needed, unless you want to add a few more packages to the existing constraints. But then the constraint solver should take that into account...

[gracjan]

Note that partial freezes are an important use case when support for more than one version of GHC is needed, simultaneously. During migration from old GHC to new GHC it is convenient to comment-out base packages (like template-haskell) while keeping constant set of versions of higher level packages.

[mietek]

@gracjan: I agree, upgrading to new GHC or Cabal package versions is an important workflow, and one which I believe could use a lot more tooling support than what is currently available.

[ttuegel]

Related to #1929.