Standardise the release process for RN OSS #17
Comments
kelset
added
🗣 Discussion
|
First, thanks for getting 0.57 rc out! (finally) I'll test building on Windows before I say more on that. Before talking about process, is there a way to test integration with Metro more ahead of time? Or I guess this is part of process, for better CI assurance of non-breakage with Metro. 0.57 is NOT the first time I've seen Metro derailing the release train, or breaking post-release because breakage in certain yet common situations wasn't tested/caught. Perhaps this should be a separate discussion, for more thorough CI testing to ensure Metro version X doesn't break with RN at master. I'm not opposed to vXX.0 become effectively the new RC. But there can be multiple rcs, how would people know at what version of vXX.N would N become actually "stable"? Not allowing multiple rcs I don't think is realistic, before vXX.1 (stable) hits. Generally as user of RN, I believe breakage in the wild of RN not even building (cough Metro cough) on .0 releases is a problem. This is fixable with better CI. And this is where I may say stupid shit because I don't know enough about RN's CI. But with this change in 0.57, is Detox just being looked at for integration into CI and/or general testing with RNTester prior to releases? That would catch new classes of RN breakage that got past the building, but actually fail with a red screen on running in any actual RN app. I've seen those more than once too on .0 releases. So please, please, please push for E2E testing prior to RN releases with something like Detox. |
|
I agree with essentially everything that has been said here and it isn't worth me repeating all the things I agree with. One thing I do want to call out is that I think it would be detrimental for the community (lack of trust / confidence) if we ship fundamentally unstable releases that aren't designated as such. If we can get rid of RCs by making every release more stable via testing or otherwise, great! Otherwise it seems like we should keep the RC branding around until we believe a release is at least usable. |
|
@fungilation let's keep the discussion focused on the release process itself, at least within this issue. Your concerns regarding Metro are valid. I should be posting a proposal to keep CI green in the next few days, and that seems like a better place to discuss how to handle breakage related to Metro. |
|
Detox testing e2e can/should be a part of the release process, no? |
|
I think the idea is that testing like that would be on CI, and on every PR / commit - not just on a release ( which is why @hramos recommended waiting on that proposal ) |
|
Again, let's keep this discussion on the general release cycle. We all know of these issues, and there is a lot of work being done behind the scenes to try and get it to a proper solution (to which the PR by Hector will be related). That said, going back to @TheSavior point on:
I fully agree with this, and personally I'd prefer to keep a 2 weeks window on an RC tag because even the CI will never be able to create all the edge cases that people managed to create while using React Native (which is great because it means that RN can be used in a TON of different ways, but at the same time it's hard to be confident). That said, yes CI will play a bigger role moving forward and we'll also try to have every PR to add a test case for itself. |
|
We used to cut releases every two weeks, and it was switched to monthly in facebook/react-native#11269. We didn't have a discussions repo back then, but if I recall correctly, @expo (cc @anp) was one of the drivers behind the switch. |
|
Thanks, @kelset for writing this up. To be honest, I think the only change that we need to implement (and are able to effectively maintain over time) is "time window" instead of what we have right now, which is strict "monthly cycle". There are many external factors that can affect the release and as time has shown us already, it can get longer than 2 weeks to cut the initial release candidate. That breaks the expectation of the userland of new releases landing around 1st of every month. At the same time, we do not want to be releasing versions that were in "RC" stage for a week, just for the sake of being on time.
After interacting with the community for more than 2 years and delivering updates about new releases, I came to a conclusion that many people do not pay attention to the release cycle as much as we do and are generally fine with waiting whatever is needed to ensure the release is as stable as possible. I don't think it ever caused any miscommunication and any additional formal process around it (e.g. monthly cycle, every 1st day of the month) will only add unnecessary pressure on the open source team that is handling releases in their free time. Let's remember that many issues that are causing delays are something we cannot fix ourselves and have to coordinate with different teams in different time zones. That means we also rely on other people free time to work on the issues. That said, I would propose we change "monthly cycle" to "monthly window", meaning that once the release is cut, at any point in a month, we wait for 30 days before it automatically gets promoted to a stable release. Being flexible, it will never make anyone feel guilty of a delay (yes, I felt guilty many times not delivering release "on time" or having to tell people we have a "delay"). With react-native-releases repository being now central place for changelog and releases status, I don't think it's confusing anymore where we are and what's the path going forward. |
|
Agreed on flexible release dates. What matters is stable releases, and we aren't talking about slowing releases to a complete crawl like once a year. |
|
One thing to add to my previous comment: in React Native repository we do have a file where we track progress and status of upcoming releases. I believe it used to serve the informational purpose for other developers. Unfortunately, it has a history of constantly getting outdated due to several delays. You can see that by looking at a planned release date for 0.57, 0.58 and past changes. |
|
Oh my I missed this notification! Yes, IIRC we proposed switching the RN biweekly release to a monthly release for a few reasons, but the main one that I recall is that it made things way simpler for synchronizing CRNA & RN release cycles. That and the fact that by that point the bi-weekly releases weren't as dense as they had previously been. It's pretty expensive to refresh the Expo/RN integration points and so we've tried to find a release cadence that acknowledges that but still moves quickly. Personally, I would lobby for setting full automation of the release as the priority rather than continuing to tweak the current process. This of course has better testing as a prerequisite (cc @TheSavior). |
|
Hey everyone, sorry for leaving this conversation a bit 'dry' in the past few months. There have been a couple evolutions on this:
But yeah overall the idea of not necessarily have a new minor each month is proving to be appreciated. |
|
+1 on not sticking to a strict monthly or biweekly cycle. It hasn't been happening in reality anyway. Release when it's ready and stable, targeting when certain milestones and sync (with React) are met. |
|
Thank you everyone (especially @kelset and @grabbou) for doing the hard work of managing releases. It's really substantial how much time you are investing and I'm super glad you are doing it, as well as you are thinking of ways to optimize the process. Since we are discussing release cadence, what does everyone think is the sweet spot between not burning out on making releases and how long the community is waiting? To estimate, how much time (in hours) per month do you spend on managing releases? How often do you think react native users upgrade and how often do you think they want to upgrade? |
|
I'd rather spend less time managing releases. Especially now that I believe RN can use more stable and less frequent patch releases. And that means further locking down dependency changes in Metro, React, etc. that tends to break on those minor releases. |
|
What do you think would be the ideal cadence? One release every 6, 8, 10 weeks or longer? |
|
Hard to say when there's currently no set cadence for major releases? "When it's done" is fine with me, with relative change in pace. |
|
There are two times of the year that releasing new code on iOS is pretty crucial:
Additionally, Apple tends to release:
The same holds (mostly) true for Android as well:
Android also sometimes releases x.1 versions in Dec, and sometimes a smaller bump around March or April. These releases often include new native APIs that need native bindings related to core UI components (new features to UIView, UITextInput, etc). Sketching this out towards a release calendar looks something like this
To me, this starts to lead us towards the fact that React Native should release alongside this schedule at the least. Target months would then be:
Releasing around the holidays is always rough, so I'd axe late Nov through Dec if at all possible, so maybe coalesce Nov and Dec into Jan to spread things out, resulting in about seven releases a year:
This could look like such, versioning-wise, perhaps leading to more stable release trains.
I understand I'm proposing this as someone who is uninvolved with the current React Native release process, and merely a consumer of RN releases, but I would say this could fit in nicely to the release schedule of consumers of RN. I'd love to hear some feedback around this! iOS Version History |
|
This assumes x/y > 0? OMG, just when I thought Flutter will forever leave us behind with their 1.0 "release" already! Great analysis of iOS/Android release patterns, for major RN releases to align with. |
|
First off, hats off for @eliperkins' table. I will save that somewhere because it's super informative. That said I agree with what @fungilation pointed out: this will be mostly useful after 1.x is reached. Also because that schedule implies that RN doesn't need any further work from keeping up to date with the native platforms, which is currently never the case 😅
I think what Mike pointed out above is still 100% on point:
Probably from 2 to 8+ hrs, from when I start cherry-picking to manual testing to CI to releasing to testing released. The main 'chokepoint' is always cherry-picking/making sure that everything works. But this is basically related to stability of master & CI, and we are already doing work on that.
Before 0.56 I think that the mantra was always to stay on latest - 1, so probably the upgrade was bimonthly or less. Now I feel that most people are either on 0.57, or on 0.55, or on super old versions.
The less possible, given all the issues with upgrading. Probably once per quarter or once per HF would be ideal I guess. That said currently it's not ideal to let too much time pass between new minor releases because, as I wrote in the first comment "master keeps moving forward at a crazy fast pace". |
|
Thanks @eliperkins, that is super useful. Based on your comment and @kelset's latest reply, I think six minor/major releases per year probably makes the most sense. @kelset @grabbou how long does it usually take for RN to catch up to latest features announced by Apple/Google? Your comments show that it takes a significant amount of time to make releases and that users are probably ok with fewer releases per year. What do you think about drafting a rough release schedule based on @eliperkins adjusted to when RN is actually ready for the latest version of SDKs/OS versions? |
|
@cpojer, @eliperkins’s comment says to me that in order to keep up with sdk and tooling changes we have to release at those times. We probably want to release more frequently than just that though to let people get new functionality and fixes, right? |
|
@TheSavior it doesn't make sense to force a release if the updates to work with the tooling changes aren't ready at that point, which is why I'm proposing to align the schedule with when we have been ready historically. Also, obviously we can always release a bit earlier or later, it doesn't have to be a hard deadline. Re fixes: we can still have patch releases any time when people feel like it, of course. |
I am probably not the best person to answer this question. Would dig internally to find the answer as Facebook's employees been usually the ones to author the necessary updates. I think one release per two months is a really reasonable promise and what we've been doing for a while now. |
It depends on which feature. As Mike said, the "main" blocker in that it needs to be in line with what happens internally at FB (ex. see how it went for Android 28 support, we had to wait for like 28.0.3 to be used internally IIRC); usually we manage to reach the support needed one month prior to the actual deadline, when there is one. If not, it's usually really slow as far as I can remember. The whole 'future of Android' issue is a good 'case' to see timing and how things move I think -> facebook/react-native#19297 |
|
Hey everyone - thank you all again for all the feedback. I'm going to close this as we have reached a decision on the "approach" we should have towards cutting any new release. Quoting @cpojer:
This goes along a side decision, which is that every new release will be paired with a dedicated blogpost which will provide an overview of the new features introduced. Side-effect of this is that we'll approach each release in a style similar to 0.57, with patch releases to "stabilise" each release. |
|
I do want “omg I can’t wait to upgrade because of X”. But note that there should be a happy medium. I don't want Apple's level of secret anticipation for 2 years or more, before I get the hardware refresh I'm waiting for. 😉 PS. CAN'T WAIT for my new 2019 iMac and iPad Air |
|
@kelset should we write down and summarize our release process somewhere based on the conversation we had here? |
|
I think this would be great to keep the communication with the community and make sure everyone understand how the process looks like. I propose we do it in I am happy to write it down as @kelset is probably busy with the "manifesto"-related work. |
|
Sounds good, please start the write up. I think it can go into a "RELEASES" or so doc on the main RN repo, what do you think? |
|
Sounds good. Adding to my TODO list, I will start writing this as I am
working on other releases in the meantime.
…On Mon, 25 Mar 2019 at 16:13 Christoph Nakazawa ***@***.***> wrote:
Sounds good, please start the write up. I think it can go into a
"RELEASES" or so doc on the main RN repo, what do you think?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#17 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ACWcxuXFIFAETIK5JqpJl8aKPRKLHQKDks5vaOd7gaJpZM4V9Yix>
.
|
|
Yeah it would be best to all of this in the Releases.md file in the main repo. I remember @turnrye wanted to change some things to it, but the "philosophy" should be there for sure. |
Intro
This discussion was born in the RN core team, but since the discussion was fairly recent and involves a really small group of devs I decided it was better to use this to move the conversation forward (since it impacts all OSS and allows to clarify how the process works).
The Gist of it
React Native as an Open Source project has many peculiarities, among which the fact that the release cycle for open source is unrelated to the internal FB usage of the library.
For this reason, the release cycle has always been decided informally by members of the core, in particular (from my understanding) @grabbou and a few other devs over time.
The timeframe between a release and the next one used to be by-weekly up to a change in late 2016, when it become monthly. Lately with the introduction of some massive changes (ex. Babel 7) the last two versions 0.56 and (currently being released) 0.57 have lagged a bit behind to this monthly cycle.
This was caused because, of course, priority is given to make sure that the release pass some basic manual testing, and until we are able to generate the
RNTesterproject and theRNTestProjectwe don't proceed to release.Why standardise
The reasoning behind creating a more "formal/structured" list of steps for a release will allow the community at large to know at any given point why and how the next version is in a certain state, and avoid misscommunication/misunderstanding among the people handling the different pieces of a release. (so, basically, to improve the overall reliability of the process)
A release is, currently, composed of a few different steps, which I list here in the order they are usually (but not necessarely) done:
v0.XX-stablev0.XX-rcXXversion so that the CI can npm publish it. This way we make it easily available for the community to test before doing the "proper" first release. Subsequent RCs can follow if some big bugs are found and fixed.v0.XX.0tag is created and the first "official" release for that version is avaiable to everyone. This timewindow is related to how buggy or easy to upgrade to the new version is.v0.XX.0has been out for 1/2 weeks thev0.XX+1-stablebranch is created and the cycle starts all over againA few extra notes:
v0.XX-rc0and the firstv0.XX+1-rc0tags, on thev0.XX-stablebranch cherry picks from the "non-stopping" master branch can happen to get some extra fixes or similar.v0.XX.0lands. This is currently mostly handled by @turnryev0.XXby the timev0.XX.0lands. This means that can be bumped even during the RC phase. This step is also necessary to keep the PRs to the doc "clean" in terms of knowing which commits of master need to wait forv0.XX+1to be merged. (this is mostly handled by @hramos and @charpeni)react-native-releasesto try and keep everyone up to date with the status of each release.Discussion points
Given that:
There are a few things to consider and discuss:
a) in what order should things happen?
Example, I think that we should not bump docs until the first RC version is released, and we should have the changelog ready for when .0 is out.
b) does it make sense to aim to a monthly release cycle?
We want to keep up the pace of master, but since the bus factor to move the release and bugfix is potentially high it may happen that things move slowly. Maybe we should consider releasing every X breaking changes or something other non-time related parameter?
c) should we remove the rc phase completely?
This was pointed out to me by @grabbou and @rozele earlier today: since we are not in 1.x it technically would make sense to start right away with a vXX.0 tag. And consider 0 as the "test bench". I am afraid this could backfire since there would not have been as much RC testing as there is currently, but again, we are not 1.x so if we can communicate properly to the community that "hey step into the .0 only if you are ready for the battle or wait for .1 for stable" it should be fine.
What do you think? Let's discuss!
The text was updated successfully, but these errors were encountered: