Add a ci test for cabal-install self-upgrade policy

[23Skidoo]

In 6c2eba7 we clarified our cabal-install support policy with respect to self-upgrade: upgrading to the latest version with cabal install cabal-install must work with all cabal-install versions released in the last three years. As discussed in #4250, we should also have a CI check that this guarantee actually holds. Testing that cabal install Cabal/ cabal-install/ succeeds with cabal-install 1.18 should be enough.

[andreabedini]

Is this the dogfood test?

[jneira]

I think the actual dogfood test builds master version of cabal-install with itself. The rest of tests builds master with one choosen cabal-install released version. This asks to build master version with the three lastest released versions (so two more than now)

[andreabedini]

Right, I understand now. I am aware this is what the website suggests but the question is:

Is this something we want to support and test?

Users could simply install a new version of cabal with ghcup or perhaps download a static binary (from here) (which AFAIK is work in progress). This would be less work on the maintainer team.

[jneira]

I think cabal install cabal-install is not the main method of upgrade cabal-install nowadays. However i think it is still widely used.
Otoh cabal users usually jumps to lastest versions quite faster (to get bug fixes! 🙂) and not having this did not suppose terrible issues afaik.
So we could rely in users and fix issues with older versions of cabal-install on demand.
Remember: if you dont test in ci, users will be the ci 😉 I think we have to take in account that when taking decissions.

So imo we can close/mark as low priority

[andreabedini]

I think cabal install cabal-install is not the main method of upgrade cabal-install nowadays.

Perhaps we can change the documentation then :) happy to close this (in this form). We can have another ticket/discussion about what install methods we want to support.

EDIT: the website mentions GHCup, Haskell Platform and self-upgrade.

[Mikolaj]

I think this would be a valuable test and doubly so because it reflects a recommended activity, but the CI may be growing too much already. Perhaps let's keep it for when we setup a separate low frequency CI that only runs overnight or only runs when PRs merge or something?

[ulysses4ever]

Let me make sure I understand it: we want a job with cabal install cabal-install that runs semi-regularly (nightly or on merge). Which host cabal do we want here? Something old, as suggested above? 1.18? GHCup has 2.4 at the oldest. HVR ppa goes to 1.16 but on old Ubuntu's, and for today's ones it's also 2.4 Any suggested method to get e.g. 1.18 on ubuntu-latest?

And installing from the local package, not from Hackage, right? (Otherwise it'd make no sense to run it in CI any time other than release.)

Tangentially, is there a nice table with a list of cabal releases and the respective dates? So that it's easy to convert "3-year window" to the oldest supported version? The table is on the Downloads, page of course.

[Mikolaj]

Let's stick with 2.4 until it doesn't work any more (and is beyond the window). Yes, from the repo using the commandline from README makes sense.

OTOH, we could have a job that uses master to install the last version available on Hackage from Hackage. That tests both the "from Hackage" instruction from README and the downgrade posibility. Both can break as the source on master evolves (the latter is less important, but may point to more important problems).