Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] Quarterly Releases #67

Merged
merged 9 commits into from
May 31, 2022
Merged

[RFC] Quarterly Releases #67

merged 9 commits into from
May 31, 2022

Conversation

driazati
Copy link
Member

@driazati driazati commented Apr 22, 2022

@driazati driazati changed the title Quarterly Releases [RFC] Quarterly Releases Apr 22, 2022
@driazati driazati marked this pull request as ready for review April 22, 2022 17:29
@areusch
Copy link
Contributor

areusch commented Apr 22, 2022

cc @apache/tvm-committers

Copy link
Member

@Mousius Mousius left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for formalising this RFC @driazati, this is a great improvement to how releases are currently done in TVM. I've highlighted some concerns but the sooner we start doing this the sooner we'll be able to figure out how to minimize the friction of releasing 😸

- RFC PR: [apache/tvm-rfcs#67](https://github.com/apache/tvm-rfcs/pull/67)

# Summary
This RFC proposes that TVM move to a quarterly release schedule. Releases would happen every 3 months or so on a schedule set well in advance, independent of individual feature development in TVM.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just clarifying, I assume that even though the features themselves are independent of the release we'd eventually want to pick a versioning scheme that reflects the changes made?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm imagining it as 0.9 -> 0.10 -> 0.11, etc until we decide to 1.0, I added some info about this in the reference section

This RFC proposes that TVM move to a quarterly release schedule. Releases would happen every 3 months or so on a schedule set well in advance, independent of individual feature development in TVM.

# Motivation
Releases are essential to the usage of TVM, especially now that are beginning to work on publishing binary packages for TVM under PyPi. Making TVM releases frequent forces the release process to become well documented and simple rather than bespoke and only achievable by a small group. Users benefit from releases by seeing that the the project is still under active development and providing an easy way to get new features. As of this RFC it has been five months since the last release. It could easily confuse new users when they expect some TVM feature that was only developed recently but is not present in the latest official release.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding "As of this RFC it has been five months since the last release", we could've released far earlier but there was some discomfort around evolving the relay.build API multiple times - with these regular releases, are we suggesting that we can evolve the API over a number of releases?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We certainly can change APIs over releases, though maintaining as much backwards compatibility and consistency as we can is good. @areusch and I discussed this specific issue and we were thinking we'd wait until that is resolved to start doing this RFC, I added another part to the reference about this issue specifically



# Guide-level explanation
TVM has [release process documentation](https://tvm.apache.org/docs/contribute/release_process.html). This RFC proposes that the release candidate vote thread be abolished in favor of a mechanical schedule where releases happen roughly every three months. A release branch will be cut, evaluated for a period of two weeks, then a release published. Publishing a release entails:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this RFC just provide a schedule for following the release process? The release candidate vote thread is part of that process and ensures we don't mistakenly ship a broken release, why do we want to abolish it as part of this rather than incorporate it into the two week review window?

Copy link
Member

@tqchen tqchen Apr 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @driazati for proposing the change and bringing up the goal of making release streamlined (which we all share).

Just to clarify this particular point(of release voting) as it is something that is related to the policy. Overall release policy is something that apache handles (extra) carefully. Release voting thread is mandatory per policy and it is not something that we can change project-wide.

The rationale is that the release voting thread brings broad community awareness and overall license requirements. A release needs at least three PMC vote to be shipped. We can however, keep the schedule streamlined and improve the overall efficiency driven by a release manager.

The original release policy guide is a good reference pt -- about the GPG key needs and where to upload, posts.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I wasn't aware of the rules on the vote thread. I think it is fine to keep in that case but move it to the end of the process, so the release manager does most of the work and testing but sends it to the PMC for approval before publishing anything

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need to explicit send to PMC as PMC members normally vote on the same public voting thread

0067-quarterly-releases.md Outdated Show resolved Hide resolved

# Drawbacks

* This requires a high level of commitment from a single individual (the release manager). If someone is not able to fill or execute this role effectively the release will be stalled.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interestingly, the Kafka document states that if noone is interested they just don't release - that seems a fair way to decide to skip the release and come back in 3 months rather than stalling it which could create the scenario we have now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that skip the release would be a good lightweight option for now. We can also do out of schedule release if necessary if there is a strong need to do so. But the goal is to keep a regular cadence where things happens most of the time

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good notes, I added a small section about this to the reference. Practically it makes sense as well, since there are always people in the loop deciding whether to proceed or not

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A project need releases so that end users have a way of consuming the work done since the last release including bug fixes, no releases means that as a community we are saying that end users don't need to consume the work done in the last 3 months including fixes for bugs and it lessens the value of releases and we leave it to people checking out random git SHAs and working from that. Not very sustainable for end users.

For the TVM community to be more end user focussed releases need to happen on a cadence rather than it being dependent on someone's interest. If there is no volunteer , maybe the PMC "volunteers" someone ?

TVM has [release process documentation](https://tvm.apache.org/docs/contribute/release_process.html). This RFC proposes that the release candidate vote thread be abolished in favor of a mechanical schedule where releases happen roughly every three months. A release branch will be cut, evaluated for a period of two weeks, then a release published. Publishing a release entails:

* Gathering and organizing release notes since the last release
* Posting a source code release on GitHub
Copy link
Member

@tqchen tqchen Apr 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be useful to clarify what is the "official release". The official release is the source code tarball posted to the TVM webpage download section(with the corresponding GPG keys that can be verified) and the apache svn repo.

The binary packages, and source code tag on github are "convenient by product" that is not part of the official release. I know it sounds a bit strange. ASF have a quite pedantic approach towards release and licensing. The main retionale is that for people who want to be extra "safe" on licensing implications, they want to go and download from the webpage -- where the source code comes clean without binary dependencies and things can be built from scratch. This pedantic view, however, is also why many organizations trust apache software (e.g. they can be applied without review)

Of course for most people the ability to quickly use and try things out is more important. So it is mainly a terminology concern, but does not mean these "convenient byproducts" are not important.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This list has both uploading the source to GitHub and Apache as a task for the release, do we just need to clear on GitHub that the officially released code can be found on the Apache site?

Copy link
Member

@tqchen tqchen Apr 26, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can do that in release note, but attaching the signature and sha is also fine in github.


* One week prior to release
* Cherry picks become limited to critical changes only
* The release manager begins building binaries and testing them against TVM's test suite
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let us add a dependency license check to quickly verify the new dependencies added since last release, and ensure license files are inplace and they are compatible with ASF policy

* One week prior to release
* Cherry picks become limited to critical changes only
* The release manager begins building binaries and testing them against TVM's test suite

Copy link
Member

@tqchen tqchen Apr 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A vote period(at least three days) would be needed on the release candidate. I would suggest keep the original schedule, feedback period(one week) and vote period(one week).

The feedbacks are based on the rc tarball(with GPG keys and sha512) to be verifiable to avoid possible errors during the release shipping(e.g. mistakenly packge in something or ignores something into the source tarball).

Changing a rc during voting period is quite cumbersome. As the original vote thread needs to be abandoned. A new rc tarball needs to be generated with an updated link, then open a new vote thread for the new rc. The feedback period would help to reduce possible problems here. A longer period would also help give general visibility to the community.

rfcs/0067-quarterly-releases.md Outdated Show resolved Hide resolved
rfcs/0067-quarterly-releases.md Show resolved Hide resolved

# Drawbacks

* This requires a high level of commitment from a single individual (the release manager). If someone is not able to fill or execute this role effectively the release will be stalled.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we should also have a deputy, who's aware about what is going on, and in the absence of the release manger can take over.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed that's a good idea but I don't think we need to explicitly codify that, if there is a problem with the release manager's time they should communicate that to the community and figure something out from there

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To help mitigate with this, several projects stagger release managers, with an outgoing release manager and an incoming shadow release manager. This spreads out the responsibility and gives an opportunity to formally train someone on release management duties.

@u99127
Copy link

u99127 commented Apr 28, 2022

@driazati - this is a really good starting point for releases and I'm very glad this is coming together.

I had a ponder last night.

A few additional points to consider for an enhancement to this RFC or make it clearer here in this RFC itself.

  • Lifecycle of a release , what happens to a release x.y.0 after the release point ? The community has 2 choices, use frugal release manager resources in making sure the next (x.y.0)+1 release happens on schedule and on time . Or continues to improve the quality of x.y.0 by fixing bugs and making sure that x.y.1 / x.y.2 keep happening - but then after a year we'll have too many releases and given where we are in the lifecycle of TVM, this may not be necessary.
  • At what point do we declare an old release something that the community will not support further ? i.e. x.y.0 is valid for 6 months or a year after which there will be no releases from the branch and the community will not take any bug fixes into the branch ?
  • Are release managers appointed for the life cycle of a 0.x release - i.e is it the responsibility to keep 0.x until its end of life ?


Much of this can be automated via GitHub Actions on the apache/tvm repo. Eventually (though maybe not for the upcoming release) these will handle all the building, testing, and publishing of releases so the job of the release manager will become simpler over time.

The release manager will use the release notes and discussions with developers to determine the next version number. Releases will continue the current versioning scheme of `major.minor.patch`, with a typical release involving a bump of the minor release version. Patch versions will be used for follow up releases onto a quarterly release, but not for the next quarter's release. The release manager will be responsible for maintaining the release until the next release, which mainly entails putting up a patch release if necessary. The release manager may also delegate this responsibility to another party if both agree.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like semver, and will likely be interpreted as semver, but is not. I would strongly encourage considering officially adopting and adhering to semver to help manage API expectations for groups trying to use TVM in production. Time-based releases don't always fit perfectly with semantic versioning, but careful consideration of API changes and additions should make adherence possible. https://semver.org

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another alternative is adopting a date-based version number such as year.quarter.minor, which will meet the needs for resolving dependencies but does not imply the strict API compatibility requirements across releases that semver has.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i agree we are not semver here. however, we are also not 1.0, so it might be worth reconsidering our versioning scheme once we do hit 1.0. I'd like to see some more infrastructure and processes developed (e.g. feature flags and a more codified set of expectations for large projects like metascheduler which are currently developed in-tree and merged in an effort-specific way) before we can commit to semver.

we could consider e.g. 0.202206.0, though it's a bit hard to parse. i don't think we should propose to rev the major version past 1.0 at this time, though.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree semver is probably too much for us right now and we should revisit at 1.0. Increasing version numbers seem simplest to think about which is what this RFC proposes and anecdotally date releases aren't as common for Python packages (though they are listed in PEP 440)

Copy link
Contributor

@areusch areusch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @driazati ! left some comments

rfcs/0067-quarterly-releases.md Show resolved Hide resolved

Much of this can be automated via GitHub Actions on the apache/tvm repo. Eventually (though maybe not for the upcoming release) these will handle all the building, testing, and publishing of releases so the job of the release manager will become simpler over time.

The release manager will use the release notes and discussions with developers to determine the next version number. Releases will continue the current versioning scheme of `major.minor.patch`, with a typical release involving a bump of the minor release version. Patch versions will be used for follow up releases onto a quarterly release, but not for the next quarter's release. The release manager will be responsible for maintaining the release until the next release, which mainly entails putting up a patch release if necessary. The release manager may also delegate this responsibility to another party if both agree.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i agree we are not semver here. however, we are also not 1.0, so it might be worth reconsidering our versioning scheme once we do hit 1.0. I'd like to see some more infrastructure and processes developed (e.g. feature flags and a more codified set of expectations for large projects like metascheduler which are currently developed in-tree and merged in an effort-specific way) before we can commit to semver.

we could consider e.g. 0.202206.0, though it's a bit hard to parse. i don't think we should propose to rev the major version past 1.0 at this time, though.

* The release manager will audit the licenses of all project dependencies to ensure they are compatible with Apache
* The release manager will open a GitHub issue announcing the release branch cut and target date for the release and state that any further inclusions in the release must be manually cherry-picked
* The release manager will create a PR targeted to merge into the release branch with the necessary changes to make a release (i.e. changing version numbers)
* Contributors that wish changes to be cherry-picked should comment on the announcement issue with the relevant PRs and commits and their reasoning. The release manager has final say on which changes should be included but should aim to be inclusive at this stage
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shall we add a note about contributors testing the release on real hardware?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added a short note about it but I don't want to expand the scope of this RFC too much, specific release testing instructions should be in https://tvm.apache.org/docs/contribute/release_process.html somewhere

rfcs/0067-quarterly-releases.md Outdated Show resolved Hide resolved
rfcs/0067-quarterly-releases.md Outdated Show resolved Hide resolved
@areusch
Copy link
Contributor

areusch commented Apr 29, 2022

@u99127 these are great points! i have some thoughts:

Lifecycle of a release , what happens to a release x.y.0 after the release point ? The community has 2 choices, use frugal release manager resources in making sure the next (x.y.0)+1 release happens on schedule and on time . Or continues to improve the quality of x.y.0 by fixing bugs and making sure that x.y.1 / x.y.2 keep happening - but then after a year we'll have too many releases and given where we are in the lifecycle of TVM, this may not be necessary.

It seems like there may be a difference between a 1.0 release and a 0.x release in terms of support. Perhaps a 0.x release is patch-supported for a short time (e.g. until the next release cycle starts)?

At what point do we declare an old release something that the community will not support further ? i.e. x.y.0 is valid for 6 months or a year after which there will be no releases from the branch and the community will not take any bug fixes into the branch ?

yeah this is a great question. it seems like right now our policy is to "support" the previous release and no further back. also something i'd expect to grow or perhaps we'd arrive at an LTS-style support policy depending on development cadence.

Are release managers appointed for the life cycle of a 0.x release - i.e is it the responsibility to keep 0.x until its end of life ?

it could be good to state that a release manager is expected to shepherd questions (though the entire set of committers and community is expected to try and answer without the need for shepherding) about the release on the forum. i'd propose we start with a shorter support period so it is not overly daunting to be a release manager.

@driazati
Copy link
Member Author

driazati commented May 9, 2022

I agree with @areusch that a lot changes pre and post 1.0. This RFC's main focus is getting code out the door often, so with that I think it makes the most sense to only support the current version and only publish a new release mid-cycle for critical bug fixes and security issues.

Ostensibly the release manager should be responsible for the release they published (and in the best case this would mean a release needs no patches and thus requires no extra work), but this could get passed off or reassigned as we move along based on who's available.

@areusch
Copy link
Contributor

areusch commented May 11, 2022

We discussed this RFC briefly this morning at the Community Meeting. Here are notes:

  • @Mousius notes that because we're 0.x version numbers, we don't need to worry too much about breaking APIs
  • @driazati notes that many of our APIs are stable and we should still think about this, but that we shouldn't spend a ton of time on those issues and should get releases out the door.

@areusch
Copy link
Contributor

areusch commented May 11, 2022

I think the main outstanding thing is the question of support here. I'd love for a few more folks to weigh in, tagging a few folks who may have ideas: @tqchen @jroesch @kparzysz-quic @u99127 @Mousius @leandron @comaniac @zhiics @Hzfengsy @ZihengJiang @yzhliu @masahi @icemelon

broadly, the question is: once we release version 0.X.0, should we discover a bug, we should likely fix it on that branch and on main and release 0.X.Y. how long shall we commit to doing that? If a bug is discovered on 0.X-1.0, will we fix it?

my vote is the following (for now, we can revisit at 1.0):

  • we agree we will fix major bugs on 0.X.Y where 0.X.Y is the latest release.
  • when we make a new minor version release e.g. 0.(X+1).0, we will stop maintaining releases starting with 0.X. at that time.
  • committers/pmcs should exercise their judgement when determining if something warrants a fix.
  • the same policy should apply on the forum--if someone asks a question on an old minor release version, it should be acceptable to ask them to reproduce it on the latest release.

@Mousius
Copy link
Member

Mousius commented May 11, 2022

my vote is the following (for now, we can revisit at 1.0):

  • we agree we will fix major bugs on 0.X.Y where 0.X.Y is the latest release.
  • when we make a new minor version release e.g. 0.(X+1).0, we will stop maintaining releases starting with 0.X. at that time.
  • committers/pmcs should exercise their judgement when determining if something warrants a fix.
  • the same policy should apply on the forum--if someone asks a question on an old minor release version, it should be acceptable to ask them to reproduce it on the latest release.

Just checking for my understanding, this would be:

  • Release 0.9.0 next
  • We find a bug, release 0.9.1
  • 0.9.1 is now the only supported release
  • 0.10.0 comes out
  • 0.10.0 is now the only supported release

Who does the patch releases? I'm assuming the nominated release person from 0.9.0 ?

@areusch
Copy link
Contributor

areusch commented May 11, 2022

@Mousius that's my suggestion. i don't know who should do it--perhaps at least the nominated person is responsible for finding someone to do it and ensuring it's done?

@driazati
Copy link
Member Author

thanks for summarizing the meeting! I agree with @areusch's points on support and have added some text about it

@Mousius
Copy link
Member

Mousius commented May 11, 2022

@Mousius that's my suggestion. i don't know who should do it--perhaps at least the nominated person is responsible for finding someone to do it and ensuring it's done?

I'd suggest we make it clear it may be required when taking the role, but equally if they find someone to swap with then that's ok 😸

@u99127
Copy link

u99127 commented May 11, 2022

I think the main outstanding thing is the question of support here. I'd love for a few more folks to weigh in, tagging a few folks who may have ideas: @tqchen @jroesch @kparzysz-quic @u99127 @Mousius @leandron @comaniac @zhiics @Hzfengsy @ZihengJiang @yzhliu @masahi @icemelon

broadly, the question is: once we release version 0.X.0, should we discover a bug, we should likely fix it on that branch and on main and release 0.X.Y. how long shall we commit to doing that? If a bug is discovered on 0.X-1.0, will we fix it?

my vote is the following (for now, we can revisit at 1.0):

  • we agree we will fix major bugs on 0.X.Y where 0.X.Y is the latest release.

Ack. I'd suggest moving bug classification in another RFC.

  • when we make a new minor version release e.g. 0.(X+1).0, we will stop maintaining releases starting with 0.X. at that time.

When we make a new minor version release 0.(x+1).0 , we do a 0.X.(Y+1) and close the branch, otherwise there might be bug fixes since 0.x.y that never see a release or the light of day.

I'd suggest that minor adjustment to the text to make it explicit about what we do even though it may be obvious :)

  • committers/pmcs should exercise their judgement when determining if something warrants a fix.

s/fix/backport

  • the same policy should apply on the forum--if someone asks a question on an old minor release version, it should be acceptable to ask them to reproduce it on the latest release.

Hmm, I'd suggest that we say all bugs / questions should be reproduced with trunk / main and the latest release rather than an unsupported release.

We probably need to update the "Releases tab" on gh to make this work.

This is looking like good small steps forward. Very good to see this coming together.

Ramana

@areusch
Copy link
Contributor

areusch commented May 12, 2022

ok cool, so i think we have these actionables:

  • note that when we do a minor release, we check the previous minor release branch and also do a revision release if there are unreleased changes there
  • make it clear that the release manager is responsible to ensure that bugs raised with the release are addressed via patch/revision release, but that such tasks are largely expected to be delegated to members of the community.

@u99127
Copy link

u99127 commented May 23, 2022

I think the main outstanding thing is the question of support here. I'd love for a few more folks to weigh in, tagging a few folks who may have ideas: @tqchen @jroesch @kparzysz-quic @u99127 @Mousius @leandron @comaniac @zhiics @Hzfengsy @ZihengJiang @yzhliu @masahi @icemelon

broadly, the question is: once we release version 0.X.0, should we discover a bug, we should likely fix it on that branch and on main and release 0.X.Y. how long shall we commit to doing that? If a bug is discovered on 0.X-1.0, will we fix it?

my vote is the following (for now, we can revisit at 1.0):

  • we agree we will fix major bugs on 0.X.Y where 0.X.Y is the latest release.

Agreed.

  • when we make a new minor version release e.g. 0.(X+1).0, we will stop maintaining releases starting with 0.X. at that time.

Agreed.

  • committers/pmcs should exercise their judgement when determining if something warrants a fix.
  • the same policy should apply on the forum--if someone asks a question on an old minor release version, it should be acceptable to ask them to reproduce it on the latest release.

I agree and I'd be stronger that the bug needs to be reproducible on trunk / main and the latest release.

How do we get this moving forward ?

R

Copy link
Member

@Mousius Mousius left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates @driazati, I think we've gotten a good amount of input here from folks and we can move forwards with this for a July release.

Will leave this open until end of week unless someone has more comments 😸

cc @hogepodge @leandron @areusch @tqchen @u99127

@areusch
Copy link
Contributor

areusch commented May 23, 2022

Great. let's move this to a PMC voting thread. I'll make one.


* Two weeks prior to release
* Cherry picks become limited to critical changes only
* The release manager begins building binaries and testing them against TVM's test suite and running on relevant hardware

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the test suite? Is it anything beyond what the Jenkins tests already do?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not currently and this RFC intentionally doesn't specify it, we can iterate on release-specific tests more as we go through the process

Comment on lines +50 to +51
Developers are heavily encouraged to use feature flags and Python API conventions (i.e. prefixing unstable or private APIs with an underscore) to enable in-progress features. It is important both for users to be able to access new or experimental behavior but also that they can get a stable interface, so these kinds of changes should be hidden behind opt-in flags.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Developers are heavily encouraged to use feature flags ... to enable in-progress features. It is important both for users to be able to access new or experimental behavior but also that they can get a stable interface, so these kinds of changes should be hidden behind opt-in flags.

Thanks @driazati, I think this specifically is a very positive move for TVM to make. I have encountered perception among developers that designs must be near-perfect before making it into TVM.

I'm happy to see explicit language around experimental feature flags which explicitly encourage iterative design. I wonder about your and others opinion on whether we need a separate RFC to formalize the process around introducing experimental changes and their path to graduation into stable releases?

Nonetheless this is great first step, thank you for outlining it.

@areusch areusch added the status: accepted RFC is accepted label May 31, 2022
@areusch
Copy link
Contributor

areusch commented May 31, 2022

Thanks all for the comments, the vote has passed and we'll merge this RFC.

@areusch areusch merged commit 70293c7 into apache:main May 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: accepted RFC is accepted
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants