Violations of MUST rules cannot be cured at all

[NYKevin]

The FAQ is incorrect on this point:

What do I do if I accidentally release a backwards incompatible change as a minor version?

As soon as you realize that you've broken the Semantic Versioning spec, fix the problem and release a new minor version that corrects the problem and restores backwards compatibility. Even under this circumstance, it is unacceptable to modify versioned releases. If it's appropriate, document the offending version and inform your users of the problem so that they are aware of the offending version.

The correct thing to do, under the current version of SemVer, is to stop claiming you comply with SemVer. Period. Here's my analysis:

According to rule 8:

Major version X (X.y.z | X > 0) MUST be incremented if any backwards incompatible changes are introduced to the public API. It MAY include minor and patch level changes. Patch and minor version MUST be reset to 0 when major version is incremented.

MUST is an absolute requirement. Any violation places the offending project entirely outside the scope of the standard, unless and until the violation ceases to exist. Here's RFC 2119 on this:

MUST This word, or the terms "REQUIRED" or "SHALL", mean that the definition is an absolute requirement of the specification.

SemVer rightly condemns changing old versions:

Once a versioned package has been released, the contents of that version MUST NOT be modified. Any modifications MUST be released as a new version.

This is a Good Thing. Old versions should not change. But it creates an infelicity here, because it is not possible to remove a violation of rule 8. The violation exists forever, so any project which has ever violated rule 8 (or 6, 7, etc.) is not SemVer compliant and will never again be SemVer compliant.

My recommendation: Weaken the MUSTs in rules 6-8 to SHOULDs, or provide an explicit exception for this circumstance. Because the course of action suggested in the FAQ is not always feasible (e.g. suppose an unnoticed compatibility issue was introduced 5 minor versions ago as an integral part of a new feature), I prefer the former approach, but a sufficiently broad leniency for old and unsupported versions would probably work just as well.

[alx9r]

I agree. Any semantic versioning standard needs to speak explicitly to the handling of versions that are later determined to have unintentionally violated the standard. The standard should speak to how the aberrant-ness of any version can be conclusively discovered.

[HansOlsson]

I agree that this is a serious problem with the specification, and change the MUSTs to SHOULDs is necessary - but I don't think it suffices. (Preferably also the one in point 3.)

Suppose a version 1.15.0 was released and then 1.16.0 broke an existing api, 1.17.0 released (compatible with 1.16.0 and without any new errors).

As this ticket indicates the entire project is then not SemVer 2.0.0 compatible; and there is no way to save it. The goal is to have a specification that actually handles this and similar realistic cases.

One sensible way of correcting it would be by releasing 1.18.0, 1.17.1 and 1.16.1 that (together with 1.15.0) are sort of consistent with the requirements, and in addition 1.17.1 is as close as possible to 1.17.0 and 1.16.1 as close as possible to 1.16.0.
It might also suffice to just release 1.18.0 - and in very rare cases one would (possibly) accept that 1.15.0 was broken and fix that (possibly even back to 1.0.0), and possibly with a 1.15.1.

Using SHOULD in SemVer would allow releasing both the incorrect versions and the corrections above, but wouldn't provide any guideline on the corrections. Basically both the error and the correction is "wrong" - and there is nothing saying that two wrongs make a right.
(SHOULD: This word, or the adjective "RECOMMENDED", mean that there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course.)

The reason that using SHOULD would not provide any guideline in itself is that versions are always compared to their immediate predecessor without skipping any version. Thus in addition to using SHOULD the SemVer-specification should also define the goal of version-consistency in a way that works if individual versions are incorrect.

One possibility would be to require that a sub-set of the versions (which as a minimum SHOULD include the latest patch release for each released version starting from 1.0.0) SHOULD be consistent with the version-comparison using the predecessor within the set; and that ensuring and documenting this sub-set MUST be a goal for the development of the Software.
(I.e. above we could state that 1.18.0, 1.17.1, 16.1, 1.15.0, .... are part of this sub-set; or possibly the rare alternative that 17.0, 1.16.0, 1.15.1, 1.14.0, ... are part of the sub-set.)

It would also be possible to handle violations of the MUST (and MUST NOT) in point 3; and in that case require a patch-version on the same minor version to resolve the ambiguity. (I.e. if there are two different 1.15.0 there should be a 1.15.1 that is unique.)

The first MUST (and MUST NOT) in point 2 can stay since 1.02.0 is simply not a version of the Software (i.e. just because the file exists with that tag doesn't make it a version).

[jwdonahue]

@NYKevin,

Nice catch! You have indeed found an inconsistency in the spec, but rather than weaken it, I think we should tighten it up by specifying exactly how to restore compliance, which should include public notification, removal of the offending release from all official publication feeds/sites and addition of one that is SemVer compliant, and one step along at the minor or patch level, as appropriate, such that anyone who happened to pull the offending version, can easily upgrade to the newer version. It should clearly state that the package/organization is not in compliance until all those steps have been completed.

Roll-backs are not as universally/completely supported by tooling as upgrades and often the offending version would still be in the local cache, so the next auto-upgrade window might simply use the internal cache. Hence the requirement for a higher version triplet than the offending version.

This is an aspect that should be covered in the numbered sections of the spec, not the FAQ.

[alexandrtovmach]

Thanks everyone for contributions, you're amazing 🎆 Did you find any consensus?

[jwdonahue]

I think everyone who commented on this, agrees the FAQ and the spec are in conflict. But we do not agree on the cure. I think there's three votes for weakening the spec from MUST to SHOULD and my one vote for keeping it tight.

---

Actually @HansOlsson's last two paragraphs fall more on the keep it tight side.

[jwdonahue]

@alexandrtovmach, spelling/grammar is not the correct tag for this issue. It is really about the semantics of the formal description in the spec and a perceived conflict between that and the FAQ.

[nnmrts]

I don't know if I as a non-contributor have a say in this, but I would vote for no, don't change these MUSTs to SHOULDs.

The semantics (lol) matter here. I don't feel like you can even really "violate" rule 8 in the sense you @NYKevin and others are describing it. Sure, the FAQ makes it seem like you are "breaking" the spec by accidentally releasing a backwards incompatible change as a minor version. But I would consider that a fixable bug in the software, not JUST a "breaking" of the spec.

The version string and the release are part of your software themselves (at least in my opinion), so making a mistake in that field should be treated the same as making one in your actual code. Better said, if you release a minor version with backwards incompatible changes, but the readme in that release still states the project follows SemVer, that's a bug. In your readme.md. So a fix would be to either remove that claim in the readme, like you suggested OR to remove the backwards incompatible changes from the next release, like the FAQ suggests. The latter fix assumes you are still following SemVer though.

This sounds weird and one could argue that it's not really the same as a SyntaxError that should be fixed or a missing feature to be added. While it definitely doesn't feel that way, it totally is when you look at a release as a collection of commits and file changes and mouseclicks and keyboard strokes up until that point, which are all part of the software release itself, and not a seperate thing like a blog post, or a website. And they all can accidentally "break" your software to a point that makes it backwards incompatible too. You would just fix that bug as well, release a new minor or patch version and go on with your life. Right? Who would really consider that a "violation" of the spec, and not just a bug?

I don't think the spec or the FAQ really agree with me on this view, but they also don't disagree. Because I think it's out of scope for a version name spec to dictate how to fix a bug.

EDIT: That said, yeah, of course, looking at it this way you could release every backwards incompatible change as a minor version and just claim that it was a bug, while claiming that you follow SemVer. Which would be true theoretically, but I think users of your software will quickly notice that bad intent. Also, why would anyone really do that anyway? So, to sum it all up, I think neither is it healthy for the spec to change these MUSTs to SHOULDs, nor is it great for the users of SemVer to force them to stop claiming they follow the spec, just because they made a mistake.