Releases

[sebmck]

For open source we should get into the habit of having regular releases on a strict schedule. This should allow some easy planning and set expectations for breaking changes ahead of time to users.

cc @bestander who has been doing an awesome job handling releases and @wycats who has thoughts on how to handle this.

[bestander]

This is the pattern that I acquired from React Native:

1. Every now and then cut the branch from current master with https://github.com/yarnpkg/yarn/blob/master/scripts/release-branch.sh
2. That script creates a 0.XY-stable branch and pushes a tag v0.XY.0
3. CI builds a tar.gz with https://github.com/yarnpkg/yarn/blob/master/scripts/build-dist.sh and deploys it to github releases https://github.com/yarnpkg/yarn/releases
4. Then I start testing it internally for FB repos and if there are any bugs I open issues here
5. Any fixes are merged to master and if they ar related to a branch stability I cherry-pick from master into that branch
6. After branch looks more stable I do npm version patch and push the new tag to the -stable branch
7. CI builds and deploys version v.XY.1
8. more tests and patches

This gives me some control over what is currently "stable" and how to track fixes important for the "stable" branch.

Happy to collaborate and improve this.

[wycats]

@bestander thanks for the brain dump -- that's really useful context.

The pattern that's worked really well for me in the past is the pattern that Rust and Ember acquired from Chrome and Firefox ("6 week releases").

To avoid some misconceptions:

• The release cycle is about major releases; bugfix releases are released on demand and frequently if needed. For Yarn, I'd expect fast-and-furious bugfix releases for a while until the bugs die down.
• Most of the thorny release management problems with quick releases are handled with rigorous use of CI and feature flags for all new features (more below). I don't think a 6-week release cycle could work without causing a lot of pain for everyone involved without feature flags.
• The RFC process is used to get buyin from the community before committing to a feature, but it doesn't need to happen before feature development begins. Features can be implemented on master behind a feature flag in tandem with working through RFCs. In fact, sometimes it's the best way to resolve unresolved questions in the RFCs.

The following is an outline of the basic strategy I've seen work in Ember and Rust. For the most part, I'm keeping it pretty abstract, but I'll call out concrete details that I use for examples.

For what it's worth, while it might initially seem like a lot of new process, I've found that by applying a little bit of targeted process to the way new features are added and releases are managed, you can take care of a lot of little annoyances in one fell swoop, and the overall level of project-management stress drops quite a bit. (I'm happy to go into more details about why that is; some of it is obvious and some is more emergent)

The Basic Strategy

For the most part, all development happens through pull requests. Pull requests are run through CI, so merging a pull request to master means that master always reflects a green build from the perspective of CI.

Feature Flags

If a pull request adds to the public surface area of Yarn, it must be wrapped in a feature flag. Ideally, all of the relevant functionality would be wrapped in the flag, but if that proves difficult, it's acceptable to wrap just the public entry point (the new flag in a command, the method or function itself, etc.).

By default, all feature flags are disabled. In Ember, we specify all active feature flags in features.json, which drives the build process, and a human-readable description in features.md. We also run all of the tests with all-features-on and all-features-off. When we're working on a particularly ambitious change (e.g. Glimmer), we also run additional builds with just that feature on or off.

There are a number of nice benefits to adding features through feature flags:

• Since everyone's working on the same codebase, it becomes much easier to identify places where your change is stomping on someone else's. Because we're using Flow, that even means that a change made in one piece of work will produce type errors in other in-progress work immediately.
• Because a single file drives the build process, it's easy to make decisions about which features to include somewhat late in the release cycle, and also to revert the inclusion of features during the beta if they become problematic (in practice, I can't remember any time that a feature that has made it into an Ember beta had to be reverted, but it's nice to know that we can, and comes up often when we decide to enable features).
• Occasionally, features become stale and end up getting abandoned. This happens rarely (because having features in the codebase that aren't going anywhere adds overhead to everyone else with no gain), but marking the relevant code clearly makes it easy to prune it out if a feature is abandoned.
• Because all new functionality is behind feature flags, canary/nightly testers can test out specific features they're interested in without pulling in unrelated (and especially incomplete) changes.

Importantly, bugfixes do not need to be implemented behind feature flags, and can be released immediately if desired. Interestingly, since we have feature flag infrastructure, Ember contributors sometimes put risky bugfixes behind feature flags to isolate them better, but it's not mandatory.

Stabilization / The "Go" Decision

Once a feature has stabilized, the team decides to enable it by default. In Ember, this means changing the feature in features.json to true. Once that flag has been flipped, it is automatically included in all builds ("all features on" and "all features off").

The Beta Release

Every six weeks, the next version of Yarn is released, including any new features that were stabilized during the previous six weeks. The beta branch is then replaced with the current state of the master branch. Because builds are driven by the features.json file, no other changes are needed.

Aside: In Ember, we also ship a new beta every week from the beta branch. Traffic on the beta branch is light; it's mostly documentation improvements and small bugfixes we didn't catch before stabilizing the feature, but we find it useful to clearly communicate the process by keeping it consistent. It would also be fine to only ship new betas if any changes actually happened.

For example, if we ship Yarn 1.3, and then during the next six weeks add and stabilize the yarn-link feature, it will be included in Yarn 1.4.

The idea behind the beta release is that it includes all of the features the team intends to ship with the next production release, and it's ready for serious testing.

The Production Release

Six weeks after the the beta release, the beta branch becomes the production branch, and the next production version is released. At the same time, the current master becomes the beta branch.

This process allows all development to happen via pull requests and the master branch, completely decoupled from decisions about what actually makes it into the releases.

In practice, after a release of getting acclimated to feature flags and automated releases, I've seen this happen in the real world: in both Ember and Rust, people work on master, and have separate discussions about what actually gets released. This also has another major benefit: it increases the pool of people who can reasonably contribute to ongoing development without having to participate in (or even understand) the decisions about what precisely is going into the next release.

To make it more concrete, here's a sample release schedule.

Date	Beta	Release
October 7, 2016	1.1-beta	1.0.0
November 18, 2016	1.2-beta	1.1.0
February 10, 2017	1.3-beta	1.2.0
March 24, 2017	1.4-beta	1.3.0
May 5, 2017	1.5-beta	1.4.0
June 16, 2017	1.6-beta	1.5.0

Bugfixes Can (And Should) Happen Anytime

The feature-flag and release process is about maintaining stability by having a good process around new surface area added to the public API.

Especially early in the lifetime of a project, and especially due to our strict compatibility requirements with npm, it makes a lot of sense to quickly release new bugfix releases as bugs are fixed.

It's also important to keep in mind that even "simple bugfixes" can have unintended consequences, and we probably want to make sure that people don't experience rapid churn where many bugfixes break other functionality, only to be fixed again by yet another bugfix. A good CI setup will probably help with this, but we should keep an eye on how people are reporting a feeling of churn.

Canary Releases

In my experience, the cleanest way to get new releases out to people who are explicitly willing to tolerate churn is to have nightly "canary" releases that always come with the latest green master.

This isn't a substitute for making sure that the production and beta releases are up to date with bugfixes, but it can provide another lever in the stability/immediacy tradeoff for tricky situations.

It's Fine if Things Aren't Perfect

I'm not aware of any project using this style of releases where everything is perfect, with every part automated, and every release precisely hitting the relevant dates. Things happen: people unexpectedly go on vacation, emergencies need an extra day or two to get right, real live humans need to write the CHANGELOG.

The key is that targeting automated, regular releases keeps the process disciplined, stable and understandable. To get close, you still have to try hard.

It also makes it very easy to communicate with users about what's happening: "Feature X is already in Canary and we expect it to be in 1.6-beta", who can become acclimated to the release rhythm and understand what to expect.

Optional: Commit Annotations

There is one part of the process that Ember does and I like quite a bit but isn't universal.

To make it easy for people to submit bugfixes to master and clearly communicate that they want the bugfix to be backported, they can annotate a commit with [BUGFIX beta] or [BUGFIX release].

An example of a recent commit that fixed an internal bug and asked for it to be backported to beta.

We initially automated the process of backporting these changes, assuming CI passes on the relevant target branch, but at some point along the way we lost the automation ("it's fine if things aren't perfect"). It's still a very useful way for contributors to communicate that they would like a commit to be backported, giving someone closer to the release process responsibility for doing the cherry-pick correctly.

As we've been around for a while, Ember's philosophy is to avoid risky backports to beta (and especially release), since there's always another release coming six weeks later, and people can use Canary in a pinch, and we're wary of breaking something else by accident. Since Yarn is a much newer project, I expect us to be more willing to aggressively backport fixes while we're still in fast-and-furious compatibility mode.

---

Whew that was long. I'm sure some of that could have been crisper, and I'm sure I've missed some details that are unique to Yarn. The above is my brain dump about how I've seen the process work in the past, and I'm very interested to hear from people whether it sounds good, whether there are aspects of it that would be uniquely hard for Yarn, and whether people have ideas to beef it up.

[wycats]

@bestander fwiw, I think what you're doing now is more-or-less in line with the braindump I just posted, except that it becomes a little more automated and regular. Let me know if there's anything about your process that doesn't fit in neatly with what I said 😄

[cpojer]

Maybe I missed this in the giant wall of text but what's the benefit of releasing minor releases on "1.x" instead of bumping a major version every 6 weeks? (bikeshed!)

[jamiebuilds]

Do you think we would have breaking changes every six weeks? And if we did, then what are we doing wrong?

[cpojer]

It's just similar to how Chrome and React and other open source projects version their projects these days.

[wycats]

Do you think we would have breaking changes every six weeks? And if we did, then what are we doing wrong?

Noooooooo.

It's just similar to how Chrome and React and other open source projects version their projects these days.

Chrome has no major versions, because you "can't break the web" (and they assume that the "breaking changes" they do are acceptable). I don't think React makes a breaking change release every 6 weeks? Right?

[bestander]

@wycats, I like your proposal.
Maybe it will be an overkill to implement in the next 2 weeks considering how fast bugs a crunched but looks like a good long term model.
IMHO switching to major versions sooner is better than doing 0.14 -> 15.0 later on.

Do you have a migration plan in mind?

[wycats]

@bestander now that we've released, I wanted to quickly revisit this.

I agree with @bestander that we should keep closing issues as fast as we get them. Once that slows down (hopefully won't be too long), I think we'll get good mileage out of the original proposal here.

I'll bump this again once the fast-and-furious pace tapers off.

[bestander]

Good idea, Yehuda

On 13 October 2016 at 00:16, Yehuda Katz notifications@github.com wrote:

@bestander https://github.com/bestander now that we've released, I
wanted to quickly revisit this.

I agree with @bestander https://github.com/bestander that we should
keep closing issues as fast as we get them. Once that slows down (hopefully
won't be too long), I think we'll get good mileage out of the original
proposal here.

I'll bump this again once the fast-and-furious pace tapers off.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#376 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/ACBdWHsQ0vztDCtudh7Bu1YiA5R1XYqMks5qzWpDgaJpZM4KCEK3
.

[olivierlacan]

I'd just like to support @wycats' idea that real live humans should write the changelog. Since there's no CHANGELOG.md file in the repo (or equivalent), I ventured over to the GitHub Releases page (less portable, but better than nothing) and it's a too bad that even minor releases like 0.15.0 include nothing other than the release number in their release/tag message.

It's fairly easy to add a commit-style message to tags with the annotate (-a) flag:

git tag -a v0.15.0

It's a bit unwieldy at times to include Markdown in tag annotations but the --cleanup=whitespace flag allows for git to stop ignoring # at the beginning of a line. And since GitHub can parse these tag annotations with Markdown, the output is much more reader-friendly:

The only thing currently missing from GitHub releases is an easy way to compare commits between one release and another, which is why a compare link within the body of the tag annotation can also be useful: v0.14.0...v0.15.0

They're obviously a lot more noisy than a proper human-curated changelog, but can help suss out changes that might have been omitted from the release notes/changelog.

I blabber on at length about this topic on http://keepachangelog.com/en/0.3.0/ if you want to long(er) form.

[luislobo]

@olivierlacan @bestander

It's not just noisy to read an automated tag diff (like v0.22.0...v0.23.2), it's very hard to follow what exactly changed. Some commits include several changes in different parts like #3071 where the short description is just: 0.23.1 cherry-picking

I'm a big fan of human-curated CHANGELOG.md file, if you get accustomed to maintain a CHANGELOG.md file, then the human-curation is distributed to all developers, being part of the responsibility of each developer to add its own line there. People that merges pull requests should require that a corresponding CHANGELOG entry is added to each PR.

You could just start with an empty one from now on, and just make sure that the people that merges PR checks that the author actually updated the changelog.

[bestander]

I think you have a good point, Luis, generated change list is hard to read. Do you want to head this initiative and help the project? It should be safe to try for one release and see if it helps.

…

On Thu, 13 Apr 2017 at 20:30, Luis Lobo Borobia ***@***.***> wrote: @olivierlacan <https://github.com/olivierlacan> @bestander <https://github.com/bestander> It's not just noisy to read an automated tag diff (like v0.22.0...v0.23.2 <v0.22.0...v0.23.2>), it's very hard to follow what exactly changed. Some commits include several changes in different parts like #3071 <#3071> where the short description is just: 0.23.1 cherry-picking I'm a big fan of human-curated CHANGELOG.md file, if you get accustomed to maintain a CHANGELOG.md file, then the human-curation is distributed to all developers, being part of the responsibility of each developer to add its own line there. People that merges pull requests should require that a corresponding CHANGELOG entry is added to each PR. You could just start with an empty one from now on, and just make sure that the people that merges PR checks that the author actually updated the changelog. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#376 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACBdWAb8yGelTCYE-Br6w5XZHYkPcMYMks5rvne5gaJpZM4KCEK3> .

[luislobo]

@bestander Sure! I've done that with other projects as well, I'll do it tonight.

[bestander]

Awesome. Awaiting for a PR with instructions.

…

On 13 April 2017 at 22:37, Luis Lobo Borobia ***@***.***> wrote: @bestander <https://github.com/bestander> Sure! I've done that with other projects as well, I'll do it tonight. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#376 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACBdWD9T1rpSBTle8bJ7kpngu4UkyFrPks5rvpV9gaJpZM4KCEK3> .