Does cabal-3.10 impose setup.Cabal < 3.12?

[andreasabel]

When trying to make cabal-3.10 build packages with custom setups on GHC 9.10.0 (alpha3), which ships with Cabal-3.12, cabal insists to build Cabal-3.10.3.0.
If I try to force using the installed Cabal, constraint solving fails.

system-filepath-0.4.14$ cabal build -w ghc-9.10.0 -c 'setup.Cabal >=3.12'
Resolving dependencies...
Error: cabal: Could not resolve dependencies:
[__0] trying: system-filepath-0.4.14 (user goal)
[__1] next goal: system-filepath:setup.Cabal (dependency of system-filepath)
[__1] rejecting: system-filepath:setup.Cabal-3.12.0.0/installed-c30b
(constraint from maximum version of Cabal used by Setup.hs requires <3.12)

The package itself does not impose an upper bound on Cabal:

custom-setup
  setup-depends: Cabal >= 1.8, base >=4.0 && <5

This is not a problem with cabal-3.11 (master), this one does not impose this upper bound.

The docs say (https://cabal.readthedocs.io/en/3.10/cabal-package.html#pkg-field-custom-setup-setup-depends):

If the field is not specified the implicit package set will be used. The package set contains packages bundled with GHC (i.e. base, bytestring) and specifically Cabal. The specific bounds are put on Cabal dependency: lower-bound is inferred from cabal-version, and the upper-bound is < 1.25.

(Ok, the "< 1.25" is definitively stale doc here.)
The docs seem to say that the upper bound comes from whatever is bundled with GHC, which should be including Cabal-3.12 (bundled with GHC 9.10 alpha3).

[andreasabel]

Ruining my CIs...: https://github.com/BNFC/bnfc/actions/runs/8800309303/job/24151220023#step:15:28

[andreasabel]

@ffaf1: This issue prevents me from evaluating Cabal-3.12 with a released cabal-install, and the following one with master:

• Wrong condition for passing --working-dir flag #9940

Not sure how I should verify relaxations to Cabal < 3.13 in my projects now...

[Bodigrim]

This is expected behavior:

cabal/cabal-install/src/Distribution/Client/ProjectPlanning.hs

Lines 1397 to 1414 in bccc59f

	-- As we can't predict the future, we also place a global upper
	-- bound on the lib:Cabal version we know how to interact with:
	--
	-- The upper bound is computed by incrementing the current major
	-- version twice in order to allow for the current version, as
	-- well as the next adjacent major version (one of which will not
	-- be released, as only "even major" versions of Cabal are
	-- released to Hackage or bundled with proper GHC releases).
	--
	-- For instance, if the current version of cabal-install is an odd
	-- development version, e.g. Cabal-2.1.0.0, then we impose an
	-- upper bound `setup.Cabal < 2.3`; if `cabal-install` is on a
	-- stable/release even version, e.g. Cabal-2.2.1.0, the upper
	-- bound is `setup.Cabal < 2.4`. This gives us enough flexibility
	-- when dealing with development snapshots of Cabal and cabal-install.
	--
	setupMaxCabalVersionConstraint =
	alterVersion (take 2) $ incVersion 1 $ incVersion 1 cabalVersion

[andreasabel]

Thanks for digging this out @Bodigrim!

Maybe one should not have search through the source code of cabal to get this information.

1. The docs (see OP) need to mention this.

2. The reason for the constraint should be clearer. Currently it says:

   (constraint from maximum version of Cabal used by Setup.hs requires <3.12)

   Better formulation:

   (the use of cabal 3.10.* limits the maximum version of Cabal used by Setup.hs to <3.12)

I am also a bit skeptical about placing this limit in the first place.

As we can't predict the future, we also place a global upper bound on the lib:Cabal version we know how to interact with

This sound to me like:

As I cannot predict whether my house will stand for 50 years, I will tear it down after 20 years.

I do not see the point of forcing pessimism into the constraint solving here.
Should things actually break with a too new Cabal, then I can always place a suitable upper bound with -c setup.Cabal < VERSION, or can't I?

[Bodigrim]

I am also a bit skeptical about placing this limit in the first place.

Given that Cabal-X is usually released a few weeks earlier than cabal-install-X, the limitation is quite annoying.

Could it be "current version + 4"? So that cabal-install-3.10 could be used to test Cabal-3.12, hoping that cabal-install-3.12 will be available in time for Cabal-3.14.

[andreasabel]

The original issue seems to be:

• cabal-install should warn if using a version of ghc that is newer than it knows about #415

However this issue only speaks about warnings which I find totally appropriate.
So we would want a warning if cabal-x builds Setup.hs with Cabal-y where y is a release version bigger than x rather than a hard constraint one cannot overwrite.

The commit that introduced the OP is c92b715. It has no tests and no doc changes may thus be simply revertible.

[Mikolaj]

Thank you for the report. I think you are right the documentation is outdated and too flimsy here. Let's discuss the proposal to revert c92b715. If agreed upon, that would go into cabal 3.14, I imagine. A PR with the revert would be useful (once our CI gets fixed).

Beyond the documentation and the proposal, is there anything else in this ticket? Any bug in cabal 3.12 or 3.10, for example?

[andreasabel]

Any bug in cabal 3.12 or 3.10, for example?

Well, I'd say the behavior I report here is a bug, others might disagree, e.g. @hvr who authored said commit.
It is certainly limiting cabal-install without known workaround.

[Mikolaj]

I should have been more precise: any bug introduced in cabal 3.10 or 3.12?

[mpickering]

I think it makes sense to have some kind of support window like this for older versions of cabal-install because there's no guarantee an old version of cabal-install will know how to interact with a ./Setup built with a new version of Cabal.

By similar reasoning, if you want to build packages which use a new version of Cabal library then you need to use a tool which knows how to communicate with that (and there's no guarantee that the interface will happen to be the same as before).

• Older versions might not pass a specific flag which is now needed for the build to work
• Older versions might pass a flag which doesn't exist anymore

Possible failures include:

• Loud confusing failures about passing unknown arguments to newer Cabal library versions.
• Silent failures, where not everything you expect to be built is built or things are built incorrectly.

Perhaps the right solution is a flag to allow power users to opt out of this constraint and venture into any unknown territory if they desire but I thin forcing general users to upgrade to something which works is a better overall experience.

[ffaf1]

Perhaps the right solution is a flag to allow power users to opt out of this constraint and venture into any unknown territory if they desire.

I think this is a very good way forward.

w/r/t broken CIs, can this be worked around. Can cabal make something immediate that eases the pain on users?

[Mikolaj]

Let me repeat the question: could we change something small in cabal 3.12 that would help in some of the cases? Or in cabal 3.14 if there's a risk that'd break things?

[Mikolaj]

See also haskell/haskell-language-server#4196 and #9632 it links to (please tell me if these are unrelated to the current issue).

[andreasabel]

@mpickering wrote:

Perhaps the right solution is a flag to allow power users to opt out of this constraint and venture into any unknown territory

Why yet another flag? cabal is already giving a loud warning whenever it is used with a new version of GHC it as not been certified for. Wouldn't the same suffice for a newer version of Cabal?

[michaelpj]

+1 for a warning. If the user hasn't set an upper bound then they're asking to use whatever new version of Cabal comes along. That may not be a good idea, but it's what they asked for! By all means warn if it's worrying, but it's quite annoying to not follow what the user stated.

[mpickering]

@Mikolaj I don't have any particular more comments to what I already made (there is no a priori reason that using a new Cabal library version will continue to work with an old cabal-install release).

@RyanGlScott It seems like a bug in haskell-ci to force you to used the installed Cabal library version?

[RyanGlScott]

It seems like a bug in haskell-ci to force you to used the installed Cabal library version?

I don't think I know enough about haskell-ci's design considerations to say for sure. That being said, given that there is (seemingly) no workaround for this bug other than to remove the use of any, I've requested that haskell-ci allow toggling the use of any in haskell-CI/haskell-ci#728.

[phadej]

Could it be "current version + 4"?

That would require to be more careful about backward compatibility of ./Setup interface.

I'd say that promptly managing the cabal-install releases should be easier.

[phadej]

I am also a bit skeptical about placing this limit in the first place.

It's like having base <5. Just remove all the upper bounds everywhere!

I don't trust that in the possible future case that cabal-install-x.y.z.w won't work with Cabal-x.y+2, the proper bug fix will be promptly released. Nor there are possibility to make a "revision" do disallow that combination, the broken cabal-install-x.y.z.w will be there, and people would need find out they need to upgrade.

[andreasabel]

I am also a bit skeptical about placing this limit in the first place.

It's like having base <5. Just remove all the upper bounds everywhere!

cabal-install-3.10 does not force base < 4.20 and shouldn't force Cabal < 3.12 either. Let the developer decide whether to take a optimistic or pessimistic approach to the possibility of breakage by future changes.

Upper bounds are needed to prevent build failures. So one can remove upper bounds for which no build failures are known. Preferably one will respond quickly to build failures by adding upper bounds through hackage revisions.

in the possible future case that cabal-install-x.y.z.w won't work with Cabal-x.y+2, the proper bug fix

The proper fix is then to add a Cabal < 3.13 bound to your setup-depends. No action needed from the maintainers of Cabal. Why should this be so different from other upper bounds? The caution will have Cabal < 3.13 anyways, the gamblers might not have it, but let them gamble!

[phadej]

Let the developer decide whether to take a optimistic or pessimistic approach to the possibility of breakage by future changes.

The cabal-install is "the developer" there. It calls ./Setup interface. If Cabal-x.y+2 changes something, cabal-install-x.y cannot know. This "internal" dependency of cabal-install on Cabal (in case of custom-setup packages) is not editable by hackage revisions.

The proper fix is then to add a Cabal < 3.13 bound to your setup-depends.

It's not as I try to explain. The cabal-install-3.14 would work without the constraint, cabal-install-3.12 won't.

[phadej]

Re haskell-ci issue. In the future, once we test GHC release alphas and candidates, it would be great if there were corresponding cabal-install release candidates so we can test that bundled ./Cabal version etc. works with build-type: Custom packages.

Ltely GHC release management has been improved, the GHC-9.10.1 was released on the date advertised few months in advance, so it should be possible to get cabal-install released close to GHC releases in the future.

[ulysses4ever]

@RyanGlScott have you considered using the prerelease advertized in the Cabal-3.12 announcement? It should work for custom setups. https://discourse.haskell.org/t/ann-cabal-3-12-0-0-released/9504#how-to-get-the-cabal-install-pre-release-3

[RyanGlScott]

I will happily use cabal-install-3.12.0.0 once it is fully supported by ghcup and haskell-ci.

[phadej]

I'd use cabal-install preleases if they were in ghcup prerelease channel

I was unaware there are cabal-install prerelease, as it didn't show up in ghcup list. I don't read discourse.

[Bodigrim]

Could it be "current version + 4"?

That would require to be more careful about backward compatibility of ./Setup interface.

...which is worth doing anyway.

I'd say that promptly managing the cabal-install releases should be easier.

One would hope so, but apparently it's not.

[andreasabel]

I'd use cabal-install preleases if they were in ghcup prerelease channel

I see this in ghcup list:

✔✔ cabal 3.10.3.0           latest,recommended                                    
✗  cabal 3.11.0.0.2024.4.19 latest-prerelease      

ghcup config:

...
url-source:
- GHCupURL
- https://raw.githubusercontent.com/haskell/ghcup-metadata/master/ghcup-prereleases-0.0.7.yaml
- https://raw.githubusercontent.com/haskell/ghcup-metadata/master/ghcup-prereleases-0.0.7.yaml
- https://raw.githubusercontent.com/haskell/ghcup-metadata/master/ghcup-prereleases-0.0.8.yaml

[andreasabel]

@phadej

It's not as I try to explain. The cabal-install-3.14 would work without the constraint, cabal-install-3.12 won't.

Ok, my model seems to be naive. I thought that it is only Setup.hs that needs the right Cabal version to build and function, but it the same library Cabal that dictates how cabal-install invokes Setup, and yes, then there is possibility of incongruence which isn't fixable either.

Still, replacing the hardwired Cabal <... constraint by a stern warning seems a possibility.

[phadej]

I tested with cabal 3.11.0.0.2024.4.19 and the Cabal-3.12.0.0 is picked up for setup-depends.

So the only "issue" I see here, is that cabal-install-3.12.0.0 is not released.

[Mikolaj]

The release process (#9885) for cabal 3.12.1.0 is now at the point where we finished backporting all planned PRs. In the interest of releasing cabal-install 3.12.1.0 ASAP and so eliminating (I hope) the immediate problem described in this issue for the time being, we need to move on to the next steps of the release process. If you are working on a PR that fixes this here ticket in a more future-proof way than just by releasing a new cabal-install, please kindly submit it for review now. Please also note we need a consensus on how to solve this issue, so the review process would need to be double-purposed to also include that discussion (but with a concrete prototype PR in hand, that's sometimes easier).

If the PR is not agreed on and doubly-reviewed by Monday morning, we are going to proceed with the release. But maybe a group of concerned user has the PR done and is finishing discussing it and forging the consensus behind the scenes. If so, it's time for the big reveal.

Thank you for the discussion in this ticket and at the open cabal devs meeting. I wasn't aware the emergency measure of releasing only Cabal the library has such dire consequences for some users, so now we are wiser and we'd probably rather choose to sync a minor cabal version with a new GHC release (with all the hackery needed for that due to a versioned API) than to stagger a major version release as we did this time. If you have any spare capacity to help with releasing cabal, that would be awesome and maybe it would enable us to just do normal releases twice a year together with GHC and not spend time on tricks and hacks instead. :) Any other contribution, e.g., reviewing PRs (https://github.com/haskell/cabal/pulls?q=is%3Apr+is%3Aopen+label%3A%22attention%3A+needs-review%22) would also help.