New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bundler release channels #12

Open
wants to merge 2 commits into
base: master
from

Conversation

Projects
None yet
5 participants
@indirect
Member

indirect commented Jul 1, 2018

As the developers of Bundler, we want to be able to continue to develop and revise Bundler into a better version of itself. As users of Bundler, we want the safest, most stable, least problematic version of Bundler at any given time.

Release channels are a way for the Bundler team to officially bless and support versions that have proven themselves to be stable. It also adds an easy and straightforward way to try the latest upcoming changes to Bundler at any time, increasing the chances that bugs can be found and fixed.

Bundler release channels allow users to opt in to the level of release stability that they want. The “canary” channel has a release from every commit that builds green. The default channel has regular releases when features and bugfixes are ready. The “stable” channel only has releases that have fixes for any issues found by the default channel. When possible, bugfixes are backported from the default channel to the stable channel.

Rendered RFC text

@indirect indirect changed the title from release channels proposal to Bundler release channels Jul 1, 2018

To use the default release channel, install the Bundler gem and use it as usual. For example, you install Bundler by running `gem install bundler`, and you install gems by running `bundle install`. The default release of Bundler will let you know anytime there is a newer default release available, but will not install it for you.
To use the stable release channel, instead install with `gem install bundler-stable`. Install gems as usually, by running `bundle install`. The stable version of Bundler will let you know anytime there is a newer stable version available, but will not install it for you. You can identify stable releases of Bundler by running `bundle install --version` and looking for `(stable)` in the output.

This comment has been minimized.

@segiddins

segiddins Jul 1, 2018

Member

I dont understand how this will work, technically speaking

This comment has been minimized.

@indirect

indirect Jul 1, 2018

Member

I think one option is shipping a bundler-stable gem that overwrites the bundle binstub to point to the bundler-stable gem instead of bundler. (Release channels are global 😛). Maybe another option is to double down on our RubyGems Bundler-loading hackery and prefer the latest stable if it is installed at all?

This comment has been minimized.

@segiddins

segiddins Jul 1, 2018

Member

I'm tempted to ask this question: would we be better off adding some support for this into rubygems

Stable releases are created by running a new rake task, `rake release:stable`. Rather than creating a new version from the latest commit, `rake release:stable` takes an existing version of Bundler as an argument, and releases that version to the stable channel. The rake task needs to check out the tag, edit the gemspec to change the name of the gem, and then build and push to RubyGems.org. Minor versions probably should not be promoted to stable until they have had at least one bugfix release or two weeks in the default channel with no significant issues.
Canary releases are created automatically from Travis CI anytime the build passes with the latest released Ruby and RubyGems versions. The canary release task edits the gemspec to change the gem name and append the first 6 characters of the commit sha to the end of the version.

This comment has been minimized.

@segiddins

segiddins Jul 1, 2018

Member

could that lead to non-monotonic version numbers ?

This comment has been minimized.

@indirect

indirect Jul 1, 2018

Member

I think we would need a timestamp-derived section of the version number as well, to ensure it would be monotonic.

This comment has been minimized.

@segiddins

segiddins Jul 1, 2018

Member

or something like git describe: v1.16.2-490-g34f909fc3

This comment has been minimized.

@hmistry

hmistry Jul 5, 2018

Contributor

That version example is not user friendly. Try to keep it a simple 3-4 digit number. 🙂

This comment has been minimized.

@indirect

indirect Jul 9, 2018

Member

Perfect, we'll append the three-digit swatch beat number (😂) at the time of the release, and then add the git sha to the end of that so we can trace to a specific commit if needed.

It would be a wonderful next step (albeit out of scope for this RFC) to ship and support one Heroku buildpack for each Bundler release channel, so that Ruby developers on Heroku can easily test each channel, give feedback, and provide bug reports _before_ Bundler releases are rolled out to all Heroku users.
Questions about breaking changes, how long to support versions, support for versions of Ruby or RubyGems, and integration with RubyGems all seem out of scope for this specific RFC.

This comment has been minimized.

@segiddins

segiddins Jul 1, 2018

Member

might want to say about the subject of an LTS version, then?

This comment has been minimized.

@indirect

indirect Jul 1, 2018

Member

Sorry, I'm not sure what you mean by this. Can you elaborate or rephrase?

This comment has been minimized.

@segiddins

segiddins Jul 1, 2018

Member

Is the subject of a Bundler long term support version on the table for the RFC, or is another topic

This comment has been minimized.

@indirect

indirect Jul 1, 2018

Member

A true LTS version would be another RFC. This RFC is purely about providing opt-in release candences, and inherits the existing support policy, which is iirc “majors get features and bugfixes for one year, and might get bugfixes for a second year but backports are not guaranteed”.

@colby-swandale

This comment has been minimized.

Member

colby-swandale commented Jul 1, 2018

I'm worried about how much more work this is going to be creating for us in terms of making releases often and keeping track of what changes are in what channel. No one on the core team is paid full-time to work on Bundler so trying to keep this idea running far into the future will be ... tough

This idea is also something that, to my knowledge, no other library is doing in Ruby. This is going to be a big new concept to introduce into the community and on selling them on the idea.

I'm also interested to know though on what will be the benefit to all this, we have a hard time asking people to test beta releases of Bundler, how will this improve on this issue?

@indirect

This comment has been minimized.

Member

indirect commented Jul 1, 2018

@colby-swandale it will definitely be more work. I'm hoping we can make it extremely automated: canary builds ship straight from travis, and anyone can opt in by running gem install bundler-canary or something like that. Stable builds will require some additional release work, but again hopefully it means simply "checkpointing" an existing release that has proven itself stable, and copying that version to the stable channel. Combining both of those things means that I hope this would produce no more than, say, 5-10% more work for us as releasers.

The canary channel makes it easy for people to test a PR that was just merged—we definitely want to make it easier to give us feedback early if something isn't working, and expecting users to clone their own copy of the git repo and install a temporary copy of Bundler is too much work.

The other way that it helps us is more or less the way that it helps everyone else using this style of releases: most users are on the default channel. Existing feature releases usually spawn a flood of miscellaneous bugs. The stable branch is a way to get regular updates without having to worry about running into those relatively-buggy releases.

I hope that the stable channel will mean that Heroku and other platforms will be willing upgrade more regularly, after the bugs have been shaken out of recent release(s). Because Heroku etc. have so many customers, they are usually 1-2 years behind on Bundler releases. This is a way to offer them what they want (relatively bug-free Bundler) based on the work that we are already doing.

# Reference-level explanation
Default releases are created using today’s release process, running `rake release`. Maybe default releases could be the same as canary releases, but with different feature flags?

This comment has been minimized.

@rafaelfranca

rafaelfranca Jul 4, 2018

It is not clear to me why gem install bundler don't install what you are calling the stable channel and gem install bundler-canary install what you are calling the default channel, care to expand the reasoning why the we have the tree channels instead of only two (canary and stable)?

This comment has been minimized.

@indirect

indirect Jul 9, 2018

Member

Sure, I can take another stab at it. Most users want a "pretty stable" version, with "pretty recent" bugfixes. The current releases try to do that, striking a midpoint between release frequency and release stability.

If we made gem install bundler install the stable channel, nothing would change from how things are today... most users would not run into bugs until the wide release, many users would run into bugs on each release, and Heroku would continue to not upgrade for months or years in an effort to stay away from new issues.

This RFC is an attempt to offer a different option, where opting in to the canary channel can either be a way to get the latest build from the master branch or be a way to always use the most recent version of Bundler that has passed all the tests in order to provide feedback.

The regular release channel then becomes the second tier of frequency/stability. We'll continue to fix known bugs before releasing, and hopefully more bugs will be caught thanks to the help of canary users.

Finally, the stable release channel is a way for most or all edge cases to be caught before we ship a release suitable for Heroku or other PaaSes to use—after a huge number of regular users have tried the release and we have ironed out any additional bugs.

In practice, this three-tier system is what we already have. Cutting edge users install Bundler from git, regular users run the latest regular release, and Heroku only installs a single carefully selected bugfix release once every 2-3 years. This RFC is about making it easier for all of those groups to participate, give us feedback, and install new releases with less stress.

This comment has been minimized.

@rafaelfranca

rafaelfranca Jul 9, 2018

That makes sense and I like the plan. How would the BUNDLED_WITH option works for cases were users have the regular channel and their machine but deploy to an infra that uses the stable channel. Would it show a warning? I believe that warning will not be actionable for the developer, so maybe it will be perceived as noise and ignored most of the time?

This comment has been minimized.

@indirect

indirect Jul 9, 2018

Member

Yeah, I'm not sure what would be best. We can either leave the warning the way it is today, and it will be noise/ignored, or we can change the warning to check for newer versions only on the specific channel that is installed. I think either one of those would probably be fine?

This comment has been minimized.

@rafaelfranca

rafaelfranca Jul 9, 2018

Checking new versions of the specific channel would be ideal, but if that requires a lot more code we can leave the warning in the way it is today.

@hmistry

This comment has been minimized.

Contributor

hmistry commented Jul 5, 2018

I share the same concerns with @colby-swandale. If I understand this RFC correctly, I personally think this will add a fair amount of overhead in keeping 3 release branches appropriately updated with what is a very small dedicated team with little funding.

From the user side, I do not see much benefit because if I want to have the latest copy, I'll use the master branch or a pre-release version. If I want the default release, then I get the latest released version. And if I want to be on a more stable version, then I'd use one of the previous minor releases with patches.

I think part of the user contention between stable and updates is timing and quality. There was a phase when there were rapid releases for bundler, I think almost every week or so which was too fast and annoying IMO because there's no chance for a version to settle out and establish what I call a "released-and-used-by-many-without-major-issues" state.

Then there were some buggy releases and I understand it will happen. To me that's a QA issue and it means it's time to review the QA process or maybe spend more time verifying before releasing. This won't mean 100% bug free releases, but the goal is to reduce the failure rate to an acceptable level which you establish. The thing with bundler is at 20+M downloads, a 1% failure means 200K fails... so find your baseline.

Regarding new features: If you want users to try out new features, then how about doing alpha releases instead. Or like Safari Technology Preview where it can be buggy and no guarantee for feature to be released in next version update.

Regarding Heroku being 1-2 years behind, I think it's probably more due to their internal processes and feeling good about a release. I would try to understand their reasons because my presumption is they might wait for the dust to settle and then start vetting a release depending on whether there's a real need to update. Basically they will always be behind as they can't afford a mistake and I don't see this RFC changing that.

These are my thoughts on the matter and there could be things I don't see but none of the rationale presented so far convinces me for the team to take on the overhead this RFC proposes. I'd rather see that time go into QA or something.

@indirect

This comment has been minimized.

Member

indirect commented Jul 9, 2018

Thanks for the feedback, @hmistry. As I mentioned to @rafaelfranca in a reply to his comment, the changes in this RFC are about smoothing the path for early adopters, which then smoothes the path for regular adopters, which then helps create a smooth path for the largest platforms that need the most stable versions possible.

Adding a canary channel will require some up-front work, but hopefully not any ongoing work. Adding a stable channel can reuse the work from canary. The added work then becomes carefully choosing an existing point release to copy to the stable channel, based on user feedback about bugs and stability. Since that should only happen once or twice per year at most, I think it's an acceptable amount of work if it gets us more user feedback and officially endorsed "stable" versions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment