Strong upperbounds seem to be the cause of major pain in the depency tool chain

[PierreR]

I might be completely wrong but I am wondering if the use of strong upperbound constraints is not causing a bunch of annoyance for the indirect usages of hnix (dhall-to-nix, ...)

For instance the upper bound on pretty-show broke the package currently on nixos unstable (it is also broken in stable releases such as 19.09 or 20.03 because of pretty-printer this time):
https://github.com/haskell-nix/hnix/blob/master/hnix.cabal#L918

Is there any reason why caret-style version operator are not used ? Would you accept a PR that does use them ?

[Anton-Latukha]

If in the world (and programming) there were no reverse compatibility issues - the Haskell ecosystem requirement for, as you say, strong upper bounds wouldn't be needed. To not have upper bounds we can just drop && < .. parts, and viola!

But the compatibility breakage is a thing - so protocol to handle them is needed.

And it seems logical that developers can/should track/be aware of breakages, be able to control them.

Haskell ecosystem and packages follow "Haskell Package Versioning Policy". And by it - the changes that breakage changes are the main cause of second point number change, aka 1.2.x -> 1.3.x, so that is why strong upper bounds in place.

AFAIK, in Cabal ^>= is just another syntactic sugar for >= .. && < .. pattern (documentation), so ^>= also places the same strong package upper bounds.

... whereas foo ^>= 1.2.3.4 asserts the positive information that “foo-1.2.3.4 is known to be compatible” and (in the absence of additional information) according to the PVP contract we can (positively) infer right away that all versions satisfying foo >= 1.2.3.4 && < 1.3 will be compatible as well.

So, foo ^>= 1.2.3.4 is a syntatic sugar for foo >= 1.2.3.4 && < 1.3.

HNix already has all dependencies adhere to the "Haskell Package Versioning Policy".

The main pain point of HNix in the upper bounds versions of dependencies - is that they are not processed/tested/accepted in the time window in which downstream stops supporting old dependency versions and so downstream builds start to fail.

---

If relying on Cabal - there is no escaping that version policy and follow it. If the project package implemented directly for other packaging language & toolchain - that it may bend the rules somewhat.

If HNix was not using cabal2nix and had Nix expression directly - inside nixpkgs with bots it could auto-roll to the degree of confidence and resiliency that builds and tests allow. It may be efficient for HNix to break-out from Cabal at some point.

---

If you are interested in some dependency management changes & making the Nix building process more modular - there were some recent messages on HNix Gitter.

[PierreR]

@Anton-Latukha Thanks for your detailed reply.

Just to confirm my (mis)understanding if pretty-show ^>= 1.9.5 had been used instead of pretty-show >= 1.9.5 && < 1.10 the effect would have been exactly the same ?

In other words, the latest pretty-show-1.10 would have been rejected all the same.

What about pretty-show ^>= 1.9 ? It would mean "I am confident this package is working with all 1.9.x versions. It might work with 1.10 but there is no guarantee".

[PierreR]

@Anton-Latukha I agree about the risk of breaking changes from 1.2.x to 1.3.x but your assertion about ^>= 1.2 still confused me because the example is using 4 digits and make a jump of 2 ;-)

[Anton-Latukha]

@PierreR

Yes.

By convention 1.9.x versions are compatible, and 1.10 - not compatible with 1.9, because that 1.10 number means it has breaking changes, and developer should check if those changes (for example API stayed the same - so external facade not changed and build & tests pass, but the behaviour of its internal functions changed - and so the program results changed, so dev at least should be aware, and better - process them).

I remember discussions like this when Cabal 2.0.0 arrived, it was a frequent topic.

From answering your issue, I derived request: #545

[Anton-Latukha]

@PierreR

If you read (mainly observe the algorithm picture) the policy - it is pretty simple and clear algorithm. And then you just need a little trust - everyone abides that policy in Haskell packaging, everyone puts the same meaning in the versioning of first two dot positions.

1.2.3.4 means it is from 1.2 compatible family. In many cases, you can use >= 1.2 instead of >= 1.2.3.4, but to safe effort, the lower bounds sometimes are not checked and just the first known version to work is used.

[PierreR]

@Anton-Latukha I understand that part.

I am still not convinced that using pretty-show ^>= 1.9 would be that bad. It would allow cabal to process 1.10 in CI and show possible point of attention. Then if the breaking changes impacts hnix I would use pretty-show >= 1.9 && < 1.10 to say I am sure 1.10 is not working.

I am really sorry to bother you with this ;-) Gitter is probably a better medium.

[Anton-Latukha]

@PierreR

^>=-assertions is used by the cabal solver to infer version constraints describing semantically compatible version ranges according to the PVP contract

It is syntactic sugar to place a strong upper bound according to the "Haskell Package Versioning Policy" (PVP).

^>= x.y is >= x.y && < x.(y+1)
^>= 1.9 already is >= 1.9 && < 1.10.

For historical reference, we can also look at old Stack discussion on this: commercialhaskell/stack#3464

[PierreR]

@Anton-Latukha Thanks for the link. I can see the discussion was dense.

I am only an external user here (and not too familiar with recent Haskell idiom).

I can't help thinking if ^>= 1.9 is merely syntactic sugar for >= 1.9 && < 1.10 "Oh my god what a huge amount of wasted time this caret operator is ..." What was the point to introduce it. I first read about this operator when it was introduced as something quite missing in the equation ...

[Anton-Latukha]

I also was puzzled by it when it was introduced.

Thank you for giving a contributing effort here.

[Profpatsch]

You can always use doJailbreak to override the boundary checks locally, until upstream fixes them. In practice, that doesn’t break anything because of Haskell’s strong types.

Refer to the Haskell documentation on how to override packages https://nixos.org/nixpkgs/manual/#haskell

[Anton-Latukha]

Yes, it is possible.

I just hoped that John would show up.

It is strange and embarrassing to advise in a project related thread to start using an interpreter and package manager by jailbreaking it in its language its package. It even can be seen as - takes courage and honesty to state helplessness. I had a similar helplessness experience from the other side of the situation while fixing the Nix installer. Now I learned to just be calm, it happens. Owner of repository and code can make anything he wants with it.