Release planning

[sjakobi]

Now that v0.10.10.1 is on Hackage, we should start planning for the next releases.

Here are some random data points (and related questions) that we can take into account:

• There are ~15 open PRs in various stages of completion that have been open for several months to years and are accumulating merge conflicts.

• 3 open PRs require a major version bump.

• The last major version bump happened in 2012.

• There are several issues that I consider bugs. (Feel free to relabel them though)

• GHC 8.12.1 is currently planned to be released on 2020-09-25.

  1. @bgamari Until which date should the next bytestring release happen so it can still get into this GHC release?
  2. Is there actually a good reason why we should aim to make another bytestring release in time for this GHC release?

[sjakobi]

Is there actually a good reason why we should aim to make another bytestring release in time for this GHC release?

One reason is that some users can't easily upgrade the boot library versions they use – at least that has been my experience with using Nix in the past few months. To give these users access to new features, we should probably try to bundle them with GHC 8.12.

[Bodigrim]

Well, then these users will just wait until GHC catches up. Not the end of the world, we do not plan any revolutionary changes.

I think it's wrong to tie bytestring to GHC release plan, unless GHC needs a particular feature. GHC release dates are far from being set in stone, so trying to keep up with them will bring all sorts of change freezes. As long as we maintain high quality of published releases, I think, bytestring can evolve mostly at its own pace.

I suggest we merge #220 and go ahead with a major bump and dropping GHC < 7.10.

Sorry, I was not clear enough above. Let's merge pending non-breaking changes + #220 and release them as 0.10.11.0 or 0.10.10.2. Then drop GHC < 7.10 and make a major bump (possibly, even to 1.0.0).

[vdukhovni]

As soon as 7.10 is dropped, I'd like to see #174 merged, with PatternSynonyms it no longer poses a compatibility problem for users of Data.ByteString.Internal (i.e. the PS constructor). And ideally in time for inclusion in GHC 8.12, I would like to see #191 also merged.

[sjakobi]

@Bodigrim I'd also like to decouple bytestring's release schedule from GHC's. Once we have accumulated some changes that ought to be released, I wouldn't want to hold off until the next GHC release.

Nevertheless, I think I'll try to pluck some low-hanging fruit in time for GHC 8.12, so users who are stuck with the bundled libraries don't have to wait longer than necessary.

I also agree that we ought to do a minor release next, and then try to flush out the breaking changes. I've created a milestone for this next minor release. Feel free to add issues and PRs to it that you feel should be included.

[sjakobi]

Since there are a bunch of unreleased changes and since there hasn't been much progress on the 0.10.12.0 milestone in the last two weeks, how about we cut the 0.10.12.0 release already?

[hsyl20]

+1

Then we could probably publish a major version. Given that GHC is going to be 9.0, it seems like a good time to merge #175 which introduces a breaking change for GHC < 8.0.

[vdukhovni]

Perhaps also #191 .

[vdukhovni]

While I haven't done any meaningful performance testing of #175 + #191, I've built code against 8.11.x + #175 + #191 and can report that they got along with each other and with various other libraries, so at least correctness and compatibility don't appear to be an issue.

[Bodigrim]

Yep, I think releasing master as 0.10.12.0 is a good idea.

[sjakobi]

I've created a release PR for 0.10.12.0: #257

While doing that, I realized that in some projects, the release preparation duty is rotated among the developers. If someone else would like to prepare one of the next releases, I wouldn't mind not doing it myself.

[Bodigrim]

@sjakobi Sorry, I'm on vacation until mid-August. If you wish, go ahead with release procedures, you have all my confidence. Otherwise, if the release can be postponed for two weeks, I'll be happy to help when I'm back.

[sjakobi]

No worries, @Bodigrim, and enjoy your vacation! :)

I'll wait until Thursday, 2020-08-06, in case anyone else wants to review #257.

[sjakobi]

0.10.12.0 is on Hackage: https://hackage.haskell.org/package/bytestring-0.10.12.0. 🚀

I forgot to prepare a proper release announcement (that could also touch upon the changes in maintainership) and I'm somewhat short on time and concentration currently. Maybe we can do one for next release.

[sjakobi]

I forgot to prepare a proper release announcement

Actually I couldn't resist posting on r/haskell: https://www.reddit.com/r/haskell/comments/i4naff/bytestring_v010120/

[sjakobi]

I have moved the remaining issues and PRs that I had originally milestoned for 0.10.12.0 to a somewhat vaguely named milestone Soon. I'd be happy if these things could be merged/addressed soon, but I don't see much pressure. (Feedback on how to appropriately use milestones and how to get nearly-finished PRs merged is welcome!)

I agree with @hsyl20 that it would be fitting to cut a major release including #175 and #181 in time for GHC 9.0. I wonder how much time we have left for this though. The first beta release of GHC 9.0 seems to be planned for 2020-09-01. Would the release need to happen before that date, @bgamari?!

Also, is anyone interested in taking on "release manager duties" for the next release? :)

[Bodigrim]

Also, is anyone interested in taking on "release manager duties" for the next release? :)

I'm happy to volunteer :)

[vdukhovni]

Are you planning to include #191?

[Bodigrim]

Yes.

[Bodigrim]

I've created a milestone for 0.11.0.0: https://github.com/haskell/bytestring/milestone/3

[vdukhovni]

With #175 merged, I think that the minimum supported version of base that can be used to build bytestring should be changed to exclude GHC 7.x and older. This should help to avoid compatibility issues should someone still using 7.x accidentally end up (lack of version bounds) using bytestring 0.11. This should also make it possible to remove a bunch of CPP guards. We should be able to assume GHC 8.0 and later. We can also drop GHC 7.x from the test matrix.

[vdukhovni]

With #175 merged, I think that the minimum supported version of base that can be used to build bytestring should be changed to exclude GHC 7.x and older[...]

This is done in #265, please review.
While we're at it, I also did some minor PS -> BS housekeeping in #264

[Bodigrim]

Discussion in #144 brings up the following question. What are we doing with PRs introducing desirable, but breaking changes?

1. Plainly reject.
2. Warehouse such PRs until there are enough changes to justify 0.12.
3. Merge into master, while maintaining a separate bytestring-0.11 branch, which accumulates backported non-breaking changes.

The previous strategy was 2, but honestly I'm not quite satisfied with the outcome. The third strategy, obviously, involves more maintenance, but seems to be the best practice.

[hsyl20]

+1 for the third strategy.

[sjakobi]

The previous strategy was 2, but honestly I'm not quite satisfied with the outcome.

@Bodigrim, can you say more about the issues you faced?

I think strategy 2 will be less work as long as the breaking PRs are few and relatively small. Once we have many and/or large breaking PRs or multiple breaking changes that depend on each other, strategy 3 will be more efficient. Alternatively, we can simply schedule a major release then and thereby try to avoid the back-porting.

[vdukhovni]

I think strategy 2 will be less work as long as the breaking PRs are few and relatively small. Once we have many and/or large breaking PRs or multiple breaking changes that depend on each other, strategy 3 will be more efficient. Alternatively, we can simply schedule a major release then and thereby try to avoid the back-porting.

The problem is that warehoused PRs tend to bitrot and require non-trivial rebasing, and authors may become difficult to reach by the time the delayed merge is scheduled.

Also contributors authoring new commits (that might also be delayed) don't have to worry about conflicts with other warehoused commits.

GHC, OpenSSL, Heimdal, SSH, ... all have stable branches and a "master" branch that merges commits as and when they're ready. I prefer option 3.

[Bodigrim]

From my perspective, this is very much about community management in open source projects. I cannot genuinely encourage people to contribute, if they cannot expect their work to be merged in a finite amount of time. For instance, I myself would rather spend time on a projects with a tighter feedback loop, where my contribution can make an impact.

If a PR is merged into master, one can manually specify git commit in cabal or stack config and benefit from it. But if a PR is warehoused, its utility is basically zero, because it is soon far behind master branch. Even if someone has a spirit to rebase it regularly, it is still impossible to use features from several PRs at once. And in a couple of years, both an author and reviewers have lost any interest and any memory of technical details, so all work should be done anew.

[sjakobi]

Good points! My assumption is that bytestring is quite different from e.g. GHC in that we'll want to include virtually everything except the breaking changes in our minor releases. In GHC, AFAIK it's mostly the bug fixes that have to be backported.

So it might be more efficient for us to keep the breaking changes on a non-default branch and to simply merge the main branch into that branch from time to time. That should be less work than cherry-picking every non-breaking change from master onto a stable branch.

[vdukhovni]

Once there are two branches, why have "master" be the stable release branch and some other branch be the future one with breaking changes? I don't think is warranted. The bug fixes should not be that many, at this point I expect most contributions to be new features, so there's not a compelling need to backport to the stable branch most of the time. So I think that the master branch should be where all new contributions get merged in the order in which they're adopted. Any that need to be backported for an interim update to a stable release can be handled as needed, but I would expect that there should be few minor update releases. Just release new versions every 6 months to (more likely perhaps) a year or so, some of which will have breaking changes and will require a major version bump.

[sjakobi]

Once there are two branches, why have "master" be the stable release branch and some other branch be the future one with breaking changes? I don't think is warranted. The bug fixes should not be that many, at this point I expect most contributions to be new features, so there's not a compelling need to backport to the stable branch most of the time.

I believe that we don't want to make major releases very often. Therefore we'll usually also want to backport new features, so they can be released with the more frequent minor releases.

[vdukhovni]

I believe that we don't want to make major releases very often. Therefore we'll usually also want to backport new features, so they can be released with the more frequent minor releases.

If so, in that case they can be merged into the stable branch shortly after they are also merged into master. Delaying either merge introduces opportunities for bitrot. In OpenSSL separate PRs are opened and generally more quickly reviewed/approved for backports into a stable branch after inclusion in master.

[Bodigrim]

OK, so the choice is:

1. Either master branch tracks stable 0.11 series, and maintainers backport features to an unstable bytestring-0.12 branch. Non-breaking PR are raised against master, and breaking ones are raised against bytestring-0.12.

2. Or master branch tracks unstable 0.12+ series, and maintainers backport features to a stable bytestring-0.11 branch. All PRs are raised against master branch.

I prefer the second option for the following reasons:

• I find it more intuitive for contributors (especially newcomers), better backed by prior art and providing more visibility of the "actual" state of the package.

• If there is a non-breaking change, merged into unstable master, which appears difficult to backport to the stable bytestring-0.11 branch, we can just skip it. Users of the stable branch would not benefit from the change, which is a bit sad, but not the end of the world.

  But if things unfold other way around, under the first option, we are in trouble. Because any change, merged into stable master, must be backported to the unstable bytestring-0.12 branch, no matter how difficult it is.

[sjakobi]

Let's try option 2 then.

[Bodigrim]

I think we can close this now as outdated and of historical interest only.