Branching Policy Draft

[arapov]

dump of the draft

Problems

• Major refactoring makes time-based releases hard, if it spans multiple releases
• We don’t have timely feature releases
• One feature shouldn’t delay another feature
• Fixes/features should be incorporated in a timely manner

Requirements (in descending order of priority)

• Time-based releases at fixed X-month intervals
• High confidence in the quality of the release
• Feature development must not delay a release
• All release branches and master should maintain linear history
• The ability to deliver features to a release at minimal latency
• Once a PR has been approved, we accept responsibility for its delivery
• Minimize the number of branches that need to be independently maintained
• The ability to develop features dependent on other features currently in the pipeline

Likely Design Decisions

There must be a branch which is ready to form the basis of a new feature release at any time

[nicowilliams]

At Sun Microsystems (RIP) we used to have a rebase-only workflow, and we had lots of project branches. Sun used this model from circa 1992 to 20010 (and probably still, its descendant products at Oracle) with Teamware and Mercurial, and I suspect now Git.

This workflow yields clean, linear history upstream, with project branches archived for posterity. This makes spelunking through history very nice and easy (the branch graph being trivial). In particular there were never any merge commits.

Here's how that workflow worked:

• We used a clone-based branching model: each project branch had its own clone of the upstream.
  Today, of course, it's just as well to use actual branches.
• Each project had its own clone/branch.
• Each developer had their own clones/branches of whatever project(s) the developer was contributing to.
• I.e., there was a branching hierarchy dev -> project -> upstream.
• Every month the project tech lead would rebase the project clone/branch onto the latest upstream bi-weekly build release. Each rebased clone/branch would have a different name.
  The project's devs would then rebase their clones/branches of the projects they contribute to.
• Eventually the project would complete, do one final rebase, and push to the upstream.

Now, for fast moving upstreams it is very important to rebase projects frequently, but if fails to (as I have when contributing to external open source projects), then it helps to have tools that can make the rebase process less painful. I have used this script (a collaboration with @vdukhovni based on original work by him) to successfully rebase a large patch across thousands of upstream commits to PostgreSQL. There are other similar tools like mergify-rebase, git-imerge, and mergify.

The tools linked above greatly help rebasing across thousands of upstream commits, searching for the commits that introduce conflicts and rebasing to those first, then rebasing further upstream. This makes the conflict resolution process much better than simply merging across thousands of commits because the most specific context (commit) causing each conflict is identified, which helps the developer resolve the conflict.

Of course, if one rebases often, then these tools are not nearly as necessary.

I highly recommend this workflow!

[arapov]

Thanks @nicowilliams! This is interesting, we will consider this.

[nicowilliams]

We used this workflow for OS/Net (the core of Solaris and Illumos) and various other codebases too. OS/Net had several thousand engineers working on it, with many projects large and small. Some projects were huge, like ZFS for example. When pushing very large projects upstream we'd lock the upstream so that no new pushes could happen until after a given project could a) rebase, b) run all their tests, c) complete its push to the upstream -- but this was fairly rare.

For open source projects I do not expect one would have the kind of project clone/branch rebasing monthly discipline, so the tools I linked are most useful.

Also, the whole clones-as-branches thing was mainly an artifact of Teamware being bad at branching, and that stuck after we moved to Mercurial, but it's not strictly a part of the process -- one can just use branches in the same repo(s).

The point of my commentary here is that a) one should not be afraid of "rebase" (many are), b) linear history works really well, c) rebasing with the right tools is easier on the developer than merging.

[vdukhovni]

@nicowilliams Thanks for those comments. I absolutely want to second those thoughts. Pretty please, with cherries on top keep the history strictly linear. Merge-based workflows are hell to understand and even harder to reconcile for someone working on side branch that gets merged with something else before they're done.

At least with a linear history, it is much easier to "replay" and rebase progressively. Let's not introduce merge commits, I don't want to see them in any project I am involved in.

[nicowilliams]

There must be a branch which is ready to form the basis of a new feature release at any time

Typically that would be the master branch in Git. If that changes, it should be very clear to contributors.

I think things that need to be defined that are very important here are:

• where should contributions land? probably the main branch, whatever its name, and there should probably be only one such branch
• backport policies? what gets backported to release branches?
• release branch naming (I propose no change here)
• versioning schemes (I propose no change here)
• release / tag naming (I propose no change here)

[mattcaswell]

Pretty please, with cherries on top keep the history strictly linear. Merge-based workflows are hell to understand and even harder to reconcile for someone working on side branch that gets merged with something else before they're done.

Absolutely! I definitely don't expect us to start using merge commits.

[ghost]

Anton will share a draft with his thoughts on this. Volunteers to help Anton to be nominated after Anton shares that info.

AI @arapov Share a draft, ask for volunteers

[arapov]

#279 must be kept in mind