Qiskit release cycle and versioning #34
Conversation
|
My first impression is that I think that bumping the major version once a year sounds about right. If semantic versioning is followed, this means only introducing breaking changes once per year. I could see two workflows that could support this. Option 1:
Option 2:
I prefer Option 1, as it means fewer branches and directs human effort and eyeballs toward the next minor release. Breaking changes can be difficult and annoying to backport, too. But I fear that there will be pressure to hold up each major release for "just one more breaking change" with a release cycle so long. Especially in that case, the deadline for having a PR ready must be well before the major version's release date. Just as Linus Torvalds hates when merge requests come in at the end of the Having thought through this more thoroughly, would we be better off to start with a 6-month cycle for a bit? One major release, then one minor release 3 months later, then back to major again 3 months after that, etc. Many, many open source projects -- from desktop environments to entire distributions -- have found 6 months to be the correct cadence. A few remaining thoughts/questions:
|
|
I'm with Jim above. I think his option 1 is probably the right way for us to try, at first at least - I don't think we have sufficient manpower or dev-team experience to attempt to manage development of two versions simultaneously (which is kind of what option 2 devolves to without a lot of effort). I'd also be happy to try a six-month cadence initially (also content to stick with yearly), though in this scenario we'd only have one minor release between majors, so I'm not certain if it'd actually buy us too much. Just to speak to the penultimate point: the formal semantic versioning spec allows deprecations to be added in minor releases (link to rule, link to faq entry). To keep the spirit of our current deprecation policy, I think we should still maintain the rule "deprecation warnings can only occur 1+ minor releases after the replacement functionality is added", and potentially strengthen the removal rule to "deprecation warnings must be emitted for a complete major version" (i.e. 1+ year). |
####-release_cycle.md
Outdated
| ### Deprecations | ||
|
|
||
| * deprecation warnings can only occur 1+ minor releases after the replacement functionality is added | ||
| * deprecation warnings must be emitted for a complete major version" (i.e. 1+ year) |
There was a problem hiding this comment.
Added from #34 (comment) @jakelishman
I would like to offer an alternative as this sounds like a lot of time to me. Introducing a deprecation warning in X.Y and not being able to remove it until X+2.0 is a huge time period of dangling code. If we keep X.3 support for 6 extra month, that should be enough to remove it in X+1.0.
There was a problem hiding this comment.
I see the trouble with this approach is that it assumes that every user will upgrade through every minor version, so will see deprecations introduced in minor releases. To me, introducing major versions means that we expect that the required upgrade path is only at the granularity of "through each major version", and so my form is what guarantees that a user will see every deprecation warning.
There was a problem hiding this comment.
For example, currently our upgrade strategy is that you can't expect to go from Terra 0.22 to 0.24 without something breaking without warning - you must go via 0.23 to ensure you've seen all warnings and had the opportunity to smoothly upgrade, and the deprecation policy also includes elements that require that there's a smooth no-error path you can take for a downstream library to support two consecutive minors, which is necessary for library authors to avoid needing precise latest-version-only pins.
I would foresee that if we add major versioning, the story ought to be "you must upgrade through each major, but you needn't install each minor", and then to keep the same spirit as our current deprecation policy, you need the deprecation warnings through an entire major release cycle.
I should be clear that what I said above is stricter than the official semantic-versioning story (which only requires a single minor release for a deprecation).
There was a problem hiding this comment.
it assumes that every user will upgrade through every minor version, so will see deprecations introduced in minor releases.
Actually, the necessary assumption is weaker: before upgrading to a new major version, the user would only need to use the final minor version of the previous release, because there, all deprecation warnings necessary for upgrading will be visible and can be fixed.
Julia is an example that illustrates both the benefits and drawbacks of this approach. Julia 0.7.0 and 1.0.0 were released within ~48 hours of each other. The only difference between them is that 0.7.0 contained compatibility (and deprecation warnings) for code that worked under 0.6, while 1.0 removed all such support. If there is ever a Julia 2.0, I expect the same thing to happen: users will be expected to use the "final" 1.X release and fix all deprecation warnings before trying the code under Julia 2.0.
The downside to this approach was that, in practice, people not intimately familiar with the release plan upgraded directly from 0.6.x to 1.0, and were met with a poor user experience: tons of code that worked on the previous release simply failed. If the goal is to get users to use a certain release before upgrading, it really does need to be emphasized in all the obvious locations, and then some.
There was a problem hiding this comment.
people not intimately familiar with the release plan upgraded directly from 0.6.x to 1.0, and were met with a poor user experience: tons of code that worked on the previous release simply failed.
I see this as expected, as a major upgrade.
I see the trouble with this approach is that it assumes that every user will upgrade through every minor version
Indeed. This is an assumption that needs to be explicit. However, it is not more obscure than the current one.
Downstream library should depend on the fairly standard ~=X.Y or ==X.*. With the extended period for X.4.* receiving critical bug fix support and not introducing new deprecations, pinning the previous major is always an option and gives a good period of time to migrate.
There was a problem hiding this comment.
My issue is that adding major versions into the mix does change the "obscurity" of the policy. I expect that code I write for one version of a library with no warnings will work in the next version of that library (now maybe with warnings), at whatever granularity that library does it major releases. Once we have majors, I see them as defining the required-upgrade granularity, as opposed to our current system where minors are that top level. That's not a requirement of semantic versioning as a spec, but that's because that spec is only for associating the scope of changes with the version number, not for any of the development/upgrade strategies that go with it.
Requiring that the 2->3 upgrade goes via 2.last and heavily publicising that is one way of dealing with my concern, but I think we've seen that we won't ever reach all our users, and I'd argue that this is a much more "obscure" requirement. People expect instability when something's versioned 0.x, but there's more expectation of stability when it's 1.x+.
edit: I think this is mostly opinion-based, so likely a resolution to this will come from a vote rather than one of us actually convincing the other.
There was a problem hiding this comment.
@garrison I actually really liked the Julia approach of having a final minor version to introduce all the deprecations prior to the major version bump. It provided a concrete path to follow as a developer to get to working code despite introducing a lot of change in 1.0.
In general, I think relying upon the final minor version as a necessary step to expose all the deprecations is a very teachable system.
There was a problem hiding this comment.
The notion of "final minor release" was introduced in 3ec912b
|
@garrison said:
Yes, the plot represent how long version has support. X.1 is supported until X.2 is released, for example. |
|
What if instead of committing to releasing a new major version each year, we simply commit to updating which version is considered "stable"? That decouples semvar signaling of version compatibility from a statement about stability / support. |
I think that's the crux of the discussion here. What Luciano and I discussed for this RFC is basically doing calver but with semver like naming. I definitely think a more traditional semver approach where we don't commit to 2.0 at a specific time and do it when we're ready has a lot of merit. In general if we go with a semver naming I would actually be more in favor of decoupling the passage of time from when we bump the major version number. However, the reason we opted for putting one release a year in the RFC was because there is a hesitancy to commit to not making any breaking API changes for > 1 year (or conversely breaking more than once a year and doing multiple major versions in a year) especially given the pace of Qiskit and quantum computing development as a whole. The two paths forward I think are we can either go fully to calver and just use date based naming so use 24.0.0 for the first release next year and commit to stability/no breaking API changes for each calendar year. This ends up leaving the rfc pretty similar to what's there now (just adding 23.0.0 to all the version strings), with an open question on how we handle deprecations/notifying users of breaking changes between each year. Also how long we continue to support a particular year's releases. The other option is use semver naming and semantics and bump the major version when we're ready to break API compatibility, not at a fixed interval and message around the stability and upgrade path between version numbers. The caveat here is we probably should put a minimum duration between major versions (similar to how the current deprecation policy sets a minimum of 3 months of support), so say at least 1 year between major versions so that we're actually committing to stability for some time, and for example not just going to release 4.0.0 in January and then 5.0.0 in April of the same year. |
I agree with option 1 too. I added it in b9fdb99 and e401ee3 |
|
A couple things I think it would be good to include (though it seems like the draft might change a lot based on the comments above):
|
This commit document expands the versioning, deprecation, and stability polices to document the policy around major versions. The previous policies were all written without any consideration for major versioning. This outlines the minimum support windows for major versions and how deprecations and breaking changes will work with regards to versioning. It also explicitly documents that Qiskit is using semantic versioning. This is a actually a change from historical releases prior to Qiskit 0.44.0 where we explicitly did not follow semantic versioning as the version number was based on the qiskit elements. This is an expanded document outlining the procedures decided on in the discussion around Qiskit/RFCs#34.
This commit document expands the versioning, deprecation, and stability polices to document the policy around major versions. The previous policies were all written without any consideration for major versioning. This outlines the minimum support windows for major versions and how deprecations and breaking changes will work with regards to versioning. It also explicitly documents that Qiskit is using semantic versioning. This is a actually a change from historical releases prior to Qiskit 0.44.0 where we explicitly did not follow semantic versioning as the version number was based on the qiskit elements. This is an expanded document outlining the procedures decided on in the discussion around Qiskit/RFCs#34.
|
I think all your comments are addressed. Have a look! |
Co-authored-by: Matthew Treinish <mtreinish@kortar.org>
Co-authored-by: Matthew Treinish <mtreinish@kortar.org>
Co-authored-by: Matthew Treinish <mtreinish@kortar.org>
Co-authored-by: Matthew Treinish <mtreinish@kortar.org>
A new versioning schema for a more clear stability cycle. SemVer with CalVer elements.