Mtl 2.3 Release Prep

[emilypi]

This is release prep for the 2.3 release of mtl.

TODO

---

• Add existing typeclass instances for SelectT
• MonadAccum (including documentation)
• MonadSelect (including documentation)
• Compatibility with transformers-0.6
• Changelog entries
• Remove all re-exports
• Version narrowing
• Cut a release

[turion]

I can send a MonadAccum PR if people would want this.

[chessai]

Edited the list to reflect a few new things that need to be done

[chessai]

Confirmed that mtl master can build with transformers-0.6

[jberryman]

Would someone be able to cut a release with what's already in master? It's a shame basic stuff like #67 has been sitting unreleased for years.

[emilypi]

@jberryman we may be able to yeah. I think now that the CLC elections are done and we're resolving procedural issues, the next steps are for @chessai and I to sync on when we have time to address this. I'm leaning towards just cutting a release currently. Stay tuned.

[phadej]

Any update on this? Not having mtl release is holding quite a lot from upgrading to transformers-0.6.

[turion]

Sorry, I'm behind with polishing MonadAccum, but I hope it's not holding up the release? I believe it's not necessary to ship it with the next version. But in case it is, I can speed up.

[phadej]

friendly fortnightly ping

[matobet]

Could we, in theory, cut a 2.3.0 release with the transformers bound bump to let the ecosystem upgrade ASAP and deliver the rest of the checklist items in subsequent 2.3.z versions?

[chessai]

This is already being done. I cut rc1 two days ago, and rc2 is coming tomorrow. Thanks everyone for the continued push!

…

On Fri, Nov 12, 2021, 19:53 Martin Beták ***@***.***> wrote: Could we, in theory, cut a 2.3.0 release with the transformers bound bump to let the ecosystem upgrade ASAP and deliver the rest of the checklist items in subsequent 2.3.z versions? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#86 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEOIX22RSMBNVNT6OKKF7LTULXAHZANCNFSM4YYDTYZA> .

[phadej]

@chessai that release candidate doesn't seem to allow transformers-0.6

[chessai]

@chessai that release candidate doesn't seem to allow transformers-0.6

Yeah, that's what rc2 is for. I forgot to bump the bounds before cutting rc1 (though I did already test building with transformers-0.6).

[phadej]

@chessai what a reason to not do #68 in mtl-2.3. I'm afraid that mtl-3 will never happen.

EDIT: I'll write a patch ASAP if we can get it into mtl-2.3.

[mrkkrp]

Is there any estimate when mtl-2.3 will be released?

[chessai]

Is there any estimate when mtl-2.3 will be released?

https://discourse.haskell.org/t/ann-release-candidate-for-mtl-2-3/3687

[phadej]

friendly ping.

[phadej]

Happy new year ping!

[mrkkrp]

Is there any estimate when mtl-2.3 will be released?

[mrkkrp]

@emilypi Is the project maintained? What is the reason 2.3 got delayed by more than 2 months while @phadej's pings are being ignored?

[tomjaguarpaw]

Hello, sorry, I didn't realise that mtl was waiting for action from me. Sorry if my inaction has been holding things up.

To be clear, my offer was not so much to "remove re-exports", but rather "decide what to do with re-exports". Part of what I would have done, were it up to me, would be to immediately release a minor version with a warning that the exports are due to be removed (see #106). My hope with that would be spread warning of the breakage slightly more widely. I understand from the discussion on the PR, @kozross, you've decided against that course of action.

With regard to actually removing the re-exports, I believe that simply involves reverting ecb525a. or equivalently, reinstating 9e1582a, so I don't see that there's anything for me to do.

[ekmett]

I think I'd be comfortable with an 8.6 floor. If you need to go below that line, you generally need to use lots of old versions of things anyways.

[kozross]

@tomjaguarpaw OK, thanks. Sorry about the confusion.

[phadej]

FYI, GHC lower-bound can be quite high: E.g. ghc-9.0 and ghc-9.2 packages depend on exceptions which depend on mtl, so on those GHCs mtl is essentially non-upgradeable (for any complicated install plan which includes e.g. ghc plugins).

OTOH, current GHC mtl master build down to GHC-7.0, so if there is no super urge to clean up all the internal CPPs etc. right now, just don't. transformers-0.6.0.4 builds with GHC-7.0, so I kind of would expect to have mtl compatible with that transformers. (old enough GHCs don't bundle mtl, so using newest mtl is simple).

(IMO, transformers-0.6 should have dropped pre-GHC-8.6 support, as it introduces QuantifiedConstraints conditionally on GHC-version, but that ship has sailed).

[kozross]

@emilypi Vis-a-vis removal of re-exports, which of these are we removing:

1. Specifically Control.Monad, Control.Monad.Fix, Data.Monoid;
2. All base re-exports;
3. All re-exports which are not our own modules; or
4. Something else?

From what I can tell of the reversions, the answer is 1, but I wanted to be sure, since the checklist you wrote doesn't specify.

@phadej The whole point of raising the floor of the GHC versions we rely upon is not to have to cart around CPP cruft, or rather, to cart around less of it. From what you're saying, it seems to be that you believe this cruft can never go because GHC somehow depends on it. I don't quite understand how this works: the boot library version of mtl is, and indeed, should be, pinned to the older version with the necessary CPP cruft to make it function, and newer GHCs should use the updated version, in which said cruft is not necessary. Am I missing something here?

[phadej]

@kozross you misunderstood me.

The most CPP in mtl is on transformers (CPS monad instances), with few __GLASGOW_HASKELL__ >= 707 (for MINIMAL pragma) and MIN_VERSION_base(4,8,0) (AMP). So you can remove the most CPP by raising the floor of transformers version, but still still supporting quite old GHCs (e.g, GHC-8.0+).

CLC has been quite conservative recently. I'd expect mtl future to be conservative as well. (supporting old GHC is not wrong, re-exporting stuff from Data.Monoid is).

[kozross]

I'm not opposed to raising the floor on the required transformers version, but this was never under discussion as far as I understand it. @emilypi and @chessai - thoughts?

I refuse to be bound by the past when it comes to improvements. Nothing is stopping anyone from using older versions of mtl for reasons of continuity, but holding improvements hostage to staying on GHC versions which are frankly archaic at this point is not something I as a maintainer plan to be held to.

[emilypi]

I support raising all floors and stopping this stupid backcompat dick-measuring contest. Please submit the PR's so I can approve them and merge them.

[chessai]

I support raising all floors and stopping this stupid backcompat dick-measuring contest. Please submit the PR's so I can approve them and merge them.

Agree

[kozross]

@emilypi and @chessai - I've sent #108 dealing with some of the points listed in this issue. It's a necessary preliminary to other work - once this lands, I'll tackle the rest.

[andreasabel]

mtl is core in the Haskell ecosystem so I expect it to be inclusive.
Ubuntu LTS 18.04 is shipping GHC 8.0 (https://repology.org/project/ghc/versions), so that would be the maximum baseline.

The immediate goal would be to release a mtl that allows to proceed to transformers-0.6 without cutting off the majority of GHC versions.

It has been argued that support for older GHCs can be achieved with older versions of mtl. Note that this isn't so easy with stackage's one-version policy. As soon as nightly switches to mtl-2.3, so must you.

One needs to balance the burden of maintaining conditionals in a library with the introduction of conditionals downstream. Throwing away information in one place upstream with require engineering equivalent information in many places downstream.

[kozross]

@andreasabel With all due respect to both Ubuntu and Stack, I simply cannot see why I should carry around their legacy, and frankly bad, decisions. GHC 8.0 is at this point something like five years old, and predates a tonne of improvements which the vast majority of the ecosystem can, and does, take for granted. Likewise, why anyone would use Stack in 2022 is completely beyond me, as literally any real benefit it ever had has long been superceded by a combination of cabal-install-3.0 and above and ghcup, even on Windows, which until fairly recently had been the only real use case.

Limiting to GHC 8.6 is not 'cutting off the majority of GHC versions' - it is cutting off a small minority, even by the most conservative estimate. It also allows us to rid ourselves a lot of maintenance cruft, which frankly should not be our responsibility to cart around because of (ultimately) downstream decisions. If Ubuntu want to package a five-year-old compiler, it's their problem to work with our decisions, not the other way around. Likewise, if Stack have a 'if we move so must you' policy of the sort you describe, it's their bad decision, and people should blame them and work around their bad choices, rather than forcing us to do it instead.

Last but not least, absolutely nothing is stopping absolutely anyone from just using the older mtl for compatibility: the older versions will not vanish magically just because 2.3 comes into existence. Them choosing to do this limits them way beyond something like transformers-0.6, but that's a problem that's pretty much outside of the purview of mtl. I simply do not accept this idea that I have to cart around the lowest common denominator of assumptions for every downstream possible forever and impede progress and improvements on that basis.

[tomjaguarpaw]

I'm quite puzzled all around here.

On the one hand, it won't hurt me personally if mtl's GHC dependency is raised all the way up to 8.6. On the other hand, making a reasonable effort to maintain a wider range of supported versions seems desirable. Perhaps the effort required is unreasonable, though.

Now, @andreasabel points out that once Stackage nightly switches to mt-2.3, anyone using nightly must also switch to mtl-2.3. But this is puzzling to me for a couple of reasons. Firstly, someone who wants to not upgrade could just not be on nightly, couldn't they? Secondly, doesn't using a particular Stackage package set require you to use a particular GHC version anyway? Can someone on nightly actually use GHC < 8.6 at all? (I'm not a Stackage or Stack user so maybe my puzzlement is due to missing some fundamental understanding.)

That said, I'm further puzzled why we can't release a non-breaking version with support for transformers-0.6. If that is a possible option then it seems unreasonable not to take it.

[emilypi]

On the other hand, making a reasonable effort to maintain a wider range of supported versions seems desirable.

Keyword reasonable here. A reasonable effort would be to maintain a fixed window of support for older GHCs. Right now, the baseline of 8.6 is 3 years old. This is a fine window for support. What is unreasonable, is to take on the burden of supporting GHCs that people do not use. The 8.6 baseline is statistically supported by the State of the Haskell survey. 6 months ago GHC <8.6 was already a diminishing ROI. Today, the returns are even less significant. The argument for older GHCs is poor and based upon speculation, and places a maintenance burden on us that has burnt out generations of CLC members in the past across many different packages.

Now, @andreasabel points out that once Stackage nightly switches to mt-2.3, anyone using nightly must also switch to mtl-2.3. But this is puzzling to me for a couple of reasons. Firstly, someone who wants to not upgrade could just not be on nightly

Yes. You opt in to using nightly. And yes, it will be a large coordinated effort ala the recent aeson-2, hashable, text-2, bytestring-0.11 upgrades etc. If people do not want to upgrade, they can pin an LTS, or they can require an upper bound in their cabal file. These are opt-in changes.

That said, I'm further puzzled why we can't release a non-breaking version with support for transformers-0.6. If that is a possible option then it seems unreasonable not to take it.

We've had this discussion, and the maintainers agreed that it would be best to just rip the bandaid off and improve the library instead of constantly kicking the ball down the road. At some point we have to stop being avoidant and accept the pain. We've decided, using our our discretion as maintainers, that this was the time.

[emilypi]

Also, please keep in mind that while Ubuntu LTS 18.04 is pinned to 8.0 in universe, this does not stop the user from installing their own locally, as they would with ghcup or stack. The reason they are pinned to that particular GHC is a mystery, as Ubuntu 20.04 jumps to 8.8 as a baseline in universe. My assumption is that whoever maintained the package stopped, for whatever reason. Either way, I'm not gonna maintain an extra 2 years of GHCs just because people don't want to add these lines to their setup:

> curl --proto '=https' --tlsv1.2 -sSf https://get-ghcup.haskell.org | sh
> ghcup install ghc 8.8.4

Rust has to do this on 18.04, and they've clearly not had trouble with success.

[tomjaguarpaw]

the maintainers agreed that it would be best to just rip the bandaid off and improve the library

Sure, I'm in favour of ripping off the bandaid ASAP. I just don't understand why, at the same time as ripping off the bandaid and releasing 2.3, there can't also be a release of 2.2 that bumps transformers.

[Kleidukos]

@tomjaguarpaw From reading the comments of the various parties here, I can judge that efforts are spread too thin, and such a compatible release would require more time and energy than is currently available.

[emilypi]

It would involve not merging any real fixes for any outstanding issues because they're hard, and just merging more CPP that we'd rip out in the next merge from @kozross the following day. That's not an exaggeration. It would literally be upon @kozross merging master into his branch. Again, we kick the can down the road. Please trust that we've made our decision, and we're going through with it, and we're going to stop derailing for conveniences. Enough is enough.

[tomjaguarpaw]

I'm further puzzled why we can't release a non-breaking version with support for transformers-0.6

After having investigated, I can answer my own question: there's at least one reason why supporting transformers-0.6 requires a major version bump in mtl: it drops ListT and ErrorT, requiring the equivalent change from mtl. That was a crucial fact that I have been missing in the discussion of all this. (Perhaps there are other reasons besides.)

[andreasabel]

Agda certainly has customers that are not software developers nor system administrators, but mathematicians (yes, the picture that this word brings to your mind is accurate). They are not following the latest developments in the Haskell ecosystem, but they want to install Agda on their old OSs. Having to reinstall GHC or upgrade the OS is a big nuisance for them.
There were always outcries when we dropped support for old GHCs, so we fixed our policy to wait for the end-of-life of Ubuntu LTS before we dropped one. The next end-of-life is April 2023 for Ubuntu LTS 18.04, and this is when we drop GHC 8.0. In one year.

So, yes, there maybe be a survey that states that only so many percent still use a GHC older than 3 years. Well, I am pretty sure that the mentioned Agda customers have not answered the survey and are not represented. I haven't answered the survey either. I do not answer surveys. They are not an objective means to data. A census would be one.

But back to topic about the release of mtl-2.3. There are already two recent RCs:

1. RC3, dropping reexports (requiring lots of fixing downstream),
2. RC4, not dropping reexports.

So, a decision is needed, but both of these release alternatives are ready, if I understood correctly.

Why then starting all over and putting more controversial stuff on the table (dropping GHCs)?

P.S.: Reconsidering, I think dropping GHCs from mtl is actually not so harmful since I can continue to use the older mtl for older GHCs. I will just be blocked from using any features of the new mtl. However what stands is that I must also support the newer mtl if I want to stay in stackage nightly (which is the gate to the stackage LTSs). If no GHCs are dropped in mtl-2.3 I can simply migrate to mtl-2.3 dropping the older versions.

[Bodigrim]

As already pointed out by @phadej above, there is nothing to win in terms of CPP from dropping GHC 8.0 support. Dropping GHC < 8.0 + dropping transformers < 0.6 is enough to get rid of any CPP in mtl, so I personally do not see much benefit in artificially reducing the support range even further immediately.

There are situations when GHC from system packages is the only bindist available. Admittedly, these cases are rare, but not extinct, e. g., if you run Ubuntu 18.04 on s390x or ppc64 architectures, there are no other options than a system package with GHC 8.0.

On the other hand, it's true that old versions of mtl are not disappearing overnight, and few people override pre-installed boot packages.

[andreasabel]

Generally speaking, the cost of restricting the scope of a new version of package X (e.g. by dropping GHCs) shows when you fix bugs in X. If you have restricted the scope in the latest release of X, you might have to back-port the fixes to older releases of X.
Not sure how relevant this is to mtl (which we expect to be bug-free).
However, in general, if I have to choose between keeping #ifs or backporting to older releases, I keep the #ifs.

[kozross]

@emilypi I think we have all necessary changelog entries at this point?

[emilypi]

Sounds like we do. I'll cut a release tomorrow.

[kozross]

Release has been cut, and is on Hackage. Closing this.