RFC proposal for semver 3.0.0 - LIBRARY version

[coldnebo]

While looking at Rails versioning, I notice that some PATCH level changes have breaking changes now. For example:

• Update Changelog entry for 5.2.4.1 to warn of breaking changes

SemVer 2.0 says:

8. Major version X (X.y.z | X > 0) MUST be incremented if any backwards incompatible changes are introduced to the public API

If Rails followed this advice, then the logical semver increment from 5.2.4 would have to be 6.x, not 5.2.4.1. But Rails 6.x already means something bigger than an incompatible API change -- it means a group of incompatible architectural changes with the previous version.

Indeed, many gems in the ecosystem face this dilemma.

Proposal: LIBRARY version

Given a version number LIBRARY.MAJOR.MINOR.PATCH, increment the:

LIBRARY version when you make incompatible architectural changes.
MAJOR version when you make incompatible API changes,
MINOR version when you add functionality in a backwards compatible manner, and
PATCH version when you make backwards compatible bug fixes.

So, for the above example, the bump could have been: 5.3.0.0

[coldnebo]

I'm not completely happy with this proposal, but I wanted to put it forth for discussion because the problem it tries to solve is real.

[ghost]

I see what you're getting at, but I think managing architectural changes is beyond SemVer's scope. It would also make newer developers more confused as to what might constitute an architectural change and a major change. I'd be interested to see what others think.

[ljharb]

If Rails followed semver, they’d be on like v37 by now.

[coldnebo]

I see what you're getting at, but I think managing architectural changes is beyond SemVer's scope. It would also make newer developers more confused as to what might constitute an architectural change and a major change. I'd be interested to see what others think.

I think to a certain extent, the LIBRARY version choice is either an architectural statement or a marketing statement. Either way, I don't think it's confusing to developers. For example, Rails developers are quite clear that changes in 6.x mean one thing and changes in 5.x mean another. Breaking changes in the API won't force them to bump that version because they've chosen it as a "brand", like "Rails 6.x"

It could be argued that semver 3.0 could simply "shift" everything over by one and stick with a usual 3 part version, but with LIBRARY.MAJOR.MINOR.

But yeah, I realize there are problems with this. It may not be fixable, but I see less and less gems trying to follow semver reliably.

[ljharb]

To me, that's reflective of a (long-standing) problem in the Ruby ecosystem, not in semver.

[cdrch]

I support the previous statements that I think this change, as written, could lead to confusion as to what counts as a sufficient change to bump LIBRARY instead of MAJOR.

However, I do like the idea of a four-part SemVer, with the new part coming first, and acting partly as an arbitrary label, be it for marketing or whatever, including splitting up technically separate projects that make use of the same codebase. The less arbitrary part of it is that it introduces a new major version 0, and therefore a period of rapid change that should be considered unstable.

So for a slightly more specific definition, LIBRARY would be bumped whenever the codebase is considered a new, sequel project, and/or when a new period of rapid, unstable development is going to occur.

I'm coming at this from the perspective of video game development, thinking of libraries and engines for development, as well as games themselves. Say you have a sequel (Shooty Game 2000) to a game (Shooty Game 1000) that uses the same core codebase, but needs to make major (breaking) changes. Under SemVer 2.0, you would bump the MAJOR version. If Shooty Game 1000 ended at v3.5.1, this would place Shooty Game 2000's start at v4.0.0. If additional breaking changes happen during development (as is likely, even with the use of -alpha/beta/rc and the like), Shooty Game 2000 could release at v5, v6, or beyond. It would provide clarity if the LIBRARY number was a marker for which project versions fell under - so Shooty Game 1000 would end at v1.3.5.1, and Shooty Game 2000 would pick up at v2.0.0.0. Shooty Game 2000 would have an initial period of rapid, unstable development at MAJOR version 0, as in the SemVar 2.0 definition, and release as v2.1.0.0. Marketing could present this as "Shooty Game 2000 v1.0.0" or whatever, but the full four-part SemVar provides clarity for all those directly interfacing with the code/API, whether that's developers or modders of the game.

There are numerous other ways to handle this, from new repositories instead of LIBRARY versions, to splitting things up into several repos, each with their own version number, to having the marketing version be totally unrelated to the in-use SemVer. However, for cases whether there's minimal change compared to the library/game as a whole, but still numerous cycles of breaking changes, the above description could prove a useful way to clarify the status of a codebase at a particular version.

All this said...I think a lot of projects would have no need for this. I wouldn't personally be opposed to having both SemVar and "SemVar Continuity" (or whatever more clever name someone else can think of) defined as options in the next version of SemVar, being MAJOR.MINOR.PATCH and LIBRARY.MAJOR.MINOR.PATCH, respectively. This goes along with some mentions of a "SemVar Lite" I saw mentioned in various issue and proposal threads elsewhere here, that (depending on the suggestion) only uses MAJOR.MINOR or MINOR.PATCH. (Maybe SemVar Lite for the former and SemVar Stable for the latter? Plus SemVar Traditional for the existing version. Enables simple abbreviations as SemVar T, SemVar C, SemVar L, and SemVar S.)

While not every project will make use of SemVar, having some additional standardized options would make it easier, I think, to shift more communities and libraries into making use of it.

[coldnebo]

To me, that's reflective of a (long-standing) problem in the Ruby ecosystem, not in semver.

I disagree. This is a problem in software in general. Java 6 means something completely different from Java 7. Bootstrap 5 means something completely different from Bootstrap 4. Python 2.7 means something completely different from Python 3.10.

What is your basis for comparison if none of these major ecosystems are using semver as written?

[coldnebo]

I support the previous statements that I think this change, as written, could lead to confusion as to what counts as a sufficient change to bump LIBRARY instead of MAJOR.

I don't understand this. Even from a game perspective (and ESPECIALLY from a game perspective), GTA 5 is completely different from GTA 4. Unity 6 is completely different from Unity 4. DirectX 8 is completely different from DirectX 12. There is no confusion among game devs about this point. Each of these is a major architectural shift. Every one of the companies behind these products goes to great lengths to establish a roadmap, plan, implement and promote a new architectural release based around a "new version" of the product. It's baked into the DNA of software development.

While it's true that changes to architecture are reducible to changes in API (thus eliminating the need for LIBRARY), imagine trying to promote the development and release of an architecture when major issues arise.... Windows 11 would be Windows 512, then 513, then 514... devs would be MORE confused when asked to compare the compatibility differences between any two such versions, like 275 vs. 518.

But if we have a LIBRARY version, e.g. like Windows 8 vs Windows 10, it becomes very simple to capture major architectural feature sets in spite of any MAJOR breaking changes that happen within that architecture.

[cdrch]

I support the previous statements that I think this change, as written, could lead to confusion as to what counts as a sufficient change to bump LIBRARY instead of MAJOR.

I don't understand this. Even from a game perspective (and ESPECIALLY from a game perspective), GTA 5 is completely different from GTA 4. Unity 6 is completely different from Unity 4. DirectX 8 is completely different from DirectX 12. There is no confusion among game devs about this point. Each of these is a major architectural shift. Every one of the companies behind these products goes to great lengths to establish a roadmap, plan, implement and promote a new architectural release based around a "new version" of the product. It's baked into the DNA of software development.

I disagree. Let me clarify my point of view, with two game series I've been a fan of: the Halo series, and the Geneforge series. (Specific knowledge beyond what I provide is unnecessary, I'm merely trying to use real-world examples.)

The Halo series (at least for the first three or so entries) was considered groundbreaking in a number of ways. Part of how this was achieved was by reinventing its engine with every entry (with the notable exception of Halo 3: ODST, which was named as such afaik due to sharing the engine with Halo 3, despite taking place simultaneously with Halo 2). Each entry had major architectural changes as a part of this. This matches with your statement.

The Geneforge series, while introducing some very interesting elements to CRPGs, certainly wasn't groundbreaking on the same scale as Halo. The engine tech follows the philosophy of its creator, Jeff Vogel, in that he seeks to continuously improve the worst aspects of the engine (and game design across his RPGs) and add a handful of new things at a time. The architecture only rarely goes through significant change all at once, and even more rarely within a series. This is very different from your statement, and this is far from the only such case.

It's interesting that you bring up Unity 6 - which does not exist. Rather, Unity dropped their existing naming scheme after 5 because they felt they had to make big architectural changes periodically (and/or gathering as many features as possible) to release with a new version, in order to be able to sell it to existing customers. They moved to their currently subscription model because it allowed them to continue to make money while being able to keep their architectural changes more gradual. Sometimes I've noted their mid-year cycle releases bring bigger changes than the yearly ones, though they generally kept breaking changes to yearly iterations rather than point releases (in other words, roughly matching SemVar in spirit). They eventually moved to treating their point releases as more akin to betas (on top of existing alpha/beta/rc designations), recommending the final release of the year as the one suitable for projects needing stability and marking them as long-term support releases.

All that to say - Unity 4 and 5 were incredibly different, but Unity 2019 is only mildly different from Unity 2020, even looking at their final versions. Determining where the line is for an "architecture" change worthy of a bump might be difficult.

To recount a specific point of yours:

Every one of the companies behind these products goes to great lengths to establish a roadmap, plan, implement and promote a new architectural release based around a "new version" of the product.

This is indeed the case with nearly all major software products, though I'd deny it is universal. However, I think you're supporting my point with this - while architecture is variable and subjective, much less subjective is "the boss says we're working on the sequel now". This permits LIBRARY to be a decision of marketing or company direction rather than trying to judge if the codebase has changed significantly enough to be called a new architecture.

While it's true that changes to architecture are reducible to changes in API (thus eliminating the need for LIBRARY), imagine trying to promote the development and release of an architecture when major issues arise.... Windows 11 would be Windows 512, then 513, then 514... devs would be MORE confused when asked to compare the compatibility differences between any two such versions, like 275 vs. 518.

But if we have a LIBRARY version, e.g. like Windows 8 vs Windows 10, it becomes very simple to capture major architectural feature sets in spite of any MAJOR breaking changes that happen within that architecture.

I absolutely agree, and is why I think your proposal (in spirit, if not in the details on which we disagree) is a very useful thing. Doubly so for less technical users who might still want to know what version they are on, but little else. I think part of the power of SemVer is letting end-users know "okay, this is just a patch/fix, so it shouldn't any issues to upgrade" and "this is a major update, so it could break programs I'm using" - though more often it might be "okay, the littlest (furthest right) number changed, so there's only a little chance something screws up what I'm doing". I think any of the software either of us referenced could make good use of having a number to let marketing do what they want with to explain things to end-users (namely, "this is a new thing!" or similar), while developers get to feel more free to use SemVer accurately. While I can't speak for modern DirectX and current Halo development, all of the other software mentioned has never made use of SemVer, in no small part because of the importance of the number that comes first to their leadership and marketing.

My apologies for long-windedness. Hopefully I made my points clearly.

[cdrch]

A more brief follow-up after chatting with others: I think I would support the term PROJECT over LIBRARY, given my preferred definitions indicate that it's more about a change in project (going to a sequel or the like) while keeping the library.

[coldnebo]

@cdrch thanks for your clarification.

"Unity 6" was my bad, but you get the point.

Really, I suppose the gist of semver is that we are trying to unambiguously communicate to devs what changes will require changes from them, vs. what changes can simply be accepted. This is crucial in package managers as I can define pessimistic version constraints like:

gem 'foo', '~> 3'

And assuming semver, can be guaranteed that none of my client code will need to change for any MINOR/PATCH updates on MAJOR 3. That is invaluable information. Without it, managing the risk of change becomes much harder to predict.

The decision of LIBRARY is more ambiguous, because we may not know exactly how many MAJOR changes equal a LIBRARY change. And I agree in some sense it is completely arbitrary. LIBRARY is not about pessimistic version control or change management per se. We have seen marketing decide to rebrand a library because "the bump in version will attract attention" rather than representing any real changes.

We have also seen some extremely CONSERVATIVE libraries that go year after year at 0.22.x, because they don't feel confident releasing that "1.x" release. In this case, I see a corruption of semver, down one decimal, as the constraints become:

gem 'neverending_beta', '~> 0.22'

which I now want to mean "no breaking changes at the MINOR level"... but that's NOT semver! Instead '0' is a statement of architectural intent: "this library is not ready for some purpose". Similarly, the statement of Rails 6.x is also a statement of architectural intent (as well as marketing): "this set of software is a significant departure from Rails 5.x". What's significant? Who knows?

Is this reducible to a pure MAJOR.MINOR.PATCH set of API changes? Sure and in theory that should be enough.

But does that mean LIBRARY is irrelevant? I'm not there yet.

There are too many forces in software development outside the control of the devs to resist this pressure. And what exactly are we resisting? Is it purely marketing and arbitrary? Or are there some grey areas where it is justified architecturally and hard to capture another way?

In either case, we already face a massive rebellion of projects claiming they support semver while corrupting its rules to support these cases. As I've shown, this is causing fundamental harm to the assumptions that devs can apply when updating versions of libraries in their code. We can no longer trust the guarantees that semver provides because people aren't really using semver!

So, either this boils down to an education/enforcement problem or there are inevitable compromises that are being made because of a different layer of value represented by LIBRARY that we can't safely ignore.

The motivation behind defining LIBRARY is so that architects, marketers and businesses can cave to these pressures ('0' for long conservative betas, or '6' for Rails new hotness) without really changing any of the version constraints in semver proper. MAJOR.MINOR.PATCH still have operational weight and we can count on them to protect change management risk. LIBRARY is merely informational and somewhat arbitrary, just a "namespace" on top of the constraint system.

[ljharb]

The project name is the mechanism for that. "Whatever 1, v1.2.3" can become "Whatever 2, v1.0.0", and there's no need to impose churn on the tooling ecosystem for that.

[coldnebo]

@ljharb yeah, that is a good point about the tooling ecosystem impact. Even adding a decimal would impose a substantial change on tooling I suppose.

So if we say that the project name is the mechanism for that, the analog in gem space would be the gem name itself:

gem 'rails_6', '~> 5' 
# vs 
gem 'rails_5', '~> 6'

Is that acceptable? If so, how do we spread the word.

EDIT: Actually, as I think more about it, this may cause issues. Specifically in the case of Rails, there is value in being able to trace the git blame across previous versions of rails. @ljharb counter-proposal (leave semver alone) would require devs to create separate repos for projects, or build the names of the gems separately from the names of the projects (tooling change).

Another signficant impact to tooling in this direction is that bundler allows gems to be loaded from git urls and branches, but there, the name of the gem would also have to coincide with the name of the url. So some tooling changes would have to be considered.

It can be done, but not as easily as I initially hoped.

[cdrch]

@cdrch thanks for your clarification.

Welcome, and please do let me know of any lack of clarity that you sense, or any points you'd like me to specifically respond to.

"Unity 6" was my bad, but you get the point.

I figured it was a typo, but it was convenient for my own rhetoric. ;)

Really, I suppose the gist of semver is that we are trying to unambiguously communicate to devs what changes will require changes from them, vs. what changes can simply be accepted.

Right. And I think that changing marking/directive changes is sufficiently important, at least sometimes, that devs should care. If there's effectively a new name for something, it's likely that you should at the least take caution when upgrading to it.

The decision of LIBRARY is more ambiguous, because we may not know exactly how many MAJOR changes equal a LIBRARY change. And I agree in some sense it is completely arbitrary. LIBRARY is not about pessimistic version control or change management per se. We have seen marketing decide to rebrand a library because "the bump in version will attract attention" rather than representing any real changes.

I'm not sure we ever know how many of any change equals a change of another level (though I see why you mention it), so I'm not sure it's important to do so here.

What's significant? Who knows?

Arguably, we never know what's significant. I've certainly done MAJOR version upgrades that didn't break anything, and I've had MINOR version upgrades that broke what I was using (due to the API being unclear and what I was doing being unintended). I've seen lots of discussions in other PRs and issues here that mention it's more about relative levels of risk than anything else, if one wanted to be reductive, and I think LIBRARY/PROJECT fits with that, in that at the least we know the developer (or their boss(es), anyways) thinks of it as a bigger change than a typical MAJOR change.

Is this reducible to a pure MAJOR.MINOR.PATCH set of API changes? Sure and in theory that should be enough.

I think SemVer is still useful with just the three, and I think some backwards compatibility can be kept by merely assuming a v1 in front of the given x.y.z of a project that ignores LIBRARY. (That is, v1.x.y.z.) LIBRARY/PROJECT is useful, but could be easily ignored by those with less need of it (much like those who will forever stay at MAJOR version 0, projects could forever stay at LIBRARY version 1).

In either case, we already face a massive rebellion of projects claiming they support semver while corrupting its rules to support these cases. As I've shown, this is causing fundamental harm to the assumptions that devs can apply when updating versions of libraries in their code. We can no longer trust the guarantees that semver provides because people aren't really using semver!

So, either this boils down to an education/enforcement problem or there are inevitable compromises that are being made because of a different layer of value represented by LIBRARY that we can't safely ignore.

Fully agree. Theoretically, x.y.z is fine. In practice, it looks more like w.x.y.z, with the w being presented in wildly inconsistent ways. Making it part of the specification would, I think, make wider adoption easier. There's additional tweaks that would make it even more appealing, if less consistent with the rest of the spec, like allowing any number in the LIBRARY/PROJECT slot, even if not starting at one or non-consecutive (like with Windows going from 8 -> 10, or Unity 2020 -> 2021). Whether that's a good idea...I'm uncertain.

LIBRARY is merely informational and somewhat arbitrary, just a "namespace" on top of the constraint system.

I'm not entirely certain if you mean that it is entirely separate and unrelated or not, so I'll clarify what I would like to see. Namely, that when you increment the version number of LIBRARY/PROJECT, it resets all the others to 0 to match with expectations.

[coldnebo]

LIBRARY is merely informational and somewhat arbitrary, just a "namespace" on top of the constraint system.

I'm not entirely certain if you mean that it is entirely separate and unrelated or not, so I'll clarify what I would like to see. Namely, that when you increment the version number of LIBRARY/PROJECT, it resets all the others to 0 to match with expectations.

EXCELLENT point! I'm wrong again. LIBRARY isn't merely informational, it changes the rules regarding how the other fields are considered.

Hmmm, ok, this point and the other raised by @ljharb have retrenched my position that LIBRARY is necessary and important.

We can try to define "project names" as @ljharb wants, but then we incur a bunch of tooling pain to support it... and to your point, it's not really consistent with semver because once the project names are forked for a new thing, then there is nothing saying that semver has to reset... which, really, it does, otherwise what kind of horrible new complexity are we asking devs to solve?

No, LIBRARY must exist and it MUST be part of the semver incrementing rules to have any weight.

[cdrch]

The project name is the mechanism for that. "Whatever 1, v1.2.3" can become "Whatever 2, v1.0.0", and there's no need to impose churn on the tooling ecosystem for that.

EDIT: Actually, as I think more about it, this may cause issues. Specifically in the case of Rails, there is value in being able to trace the git blame across previous versions of rails. @ljharb counter-proposal (leave semver alone) would require devs to create separate repos for projects, or build the names of the gems separately from the names of the projects (tooling change).

Another signficant impact to tooling in this direction is that bundler allows gems to be loaded from git urls and branches, but there, the name of the gem would also have to coincide with the name of the url. So some tooling changes would have to be considered.

It can be done, but not as easily as I initially hoped.

I think @coldnebo covers what would've been my main response to @ljharb - sometimes, it's just really useful to keep sequels of a project (or the equivalent) in the same repo, and existing SemVer would lead to having two v1.0.0 in the same repo eventually, which not only causes confusion but also breaks existing tooling for versioning.

To minimize harm done to existing projects happy with SemVar as-is, I could easily see the fourth part being optional, and assuming it to be v1 if it is absent. I think that would make the specification still work?

[coldnebo]

So, in a sense, @cdrch that is what is currently happening in the wild.

People are looking at the core of semver and seeing that defining MAJOR.MINOR.PATCH is good, and people find value in that.

Where we get confusion is that people want to redefine that pattern such that it can be "shifted" up or down precision levels.

Hence, if I have an eternal beta, I could use a "right-shift": 0.MAJOR.MINOR.PATCH

For Rails 6, I could also use a "right-shift": 6.MAJOR.MINOR.PATCH

I'm not sure this is palatable either, because we would still need rules about how a 4-place system would increment and those would be equivalent to the LIBRARY proposal.

You also raised good points about how we may never know whether a MAJOR/MINOR/PATCH update actually meets our criteria until AFTER we release a library and find a bug that breaks people. I'm not sure that question is in scope, but it impacts how we think about the guarantees that semver wants to give us.

[cdrch]

One more clarification that's I've been referring to but that hasn't been defined: I would suggest LIBRARY/PROJECT start at v1. This is because of incrementing rule inconsistencies if it starts at version 0.

Start at v0 example:

1. Start coding Project Alpha at v0.0.1.0.
2. Continue with v0.0.1.1, v0.0.2.0, etc.
3. Finish and release Project Alpha, taking it to v1.1.0.0.
4. Last update of Project Alpha is at v1.3.4.2.
5. Start Project Beta ("it makes sense, marketing told me"). It doesn't increment LIBRARY/PROJECT, so we're at v1.4.0.0, or optionally just v1.3.5.2 if it makes sense under SemVar. This is confusing because v1 covers both Alpha and Beta, two different projects. v2 (or vBeta, if you will) isn't applied until after Beta is released, which is less useful.
   Or:
6. Start Project Beta ("no really, Greek letters are cool in ad campaigns"). It increments LIBRARY/PROJECT to be consistent with being a new project, so it's now v2.0.0.0 (or v2.0.1.0). When it releases, it'll be bumped to v3.0.0.0, which is not useful at all. (Except for those companies that really like odd and even numbers having different meanings.)

Start at v1 example:

1. Start coding Project Alpha at v1.0.1.0.
2. Continue with v1.0.1.1, v1.0.2.0, etc.
3. Finish and release Project Alpha, taking it to v1.1.0.0.
4. Last update of Project Alpha is at v1.3.4.2.
5. Start Project Beta at v2.0.1.0 (or v2.0.0.0). Lack of confusion in the LIBRARY/PROJECT place, and you know the public release is when major goes from 0 -> 1.

So I think starting LIBRARY/PROJECT at v1 (and assuming it's at v1 for projects not specifying all 4) makes the most sense.

Additionally, I like that the fourth number up front still provides, logically, a MAJOR version 0 to indicate rapid change with each new project. (Just now in the form w.0.y.z.)

EDIT: No big response to your most recent post, coldnebo, I definitely agree. Going to be thinking on a formal definition that minimizes harm and maximizes the usefulness of it.

[coldnebo]

One more clarification that's I've been referring to but that hasn't been defined: I would suggest LIBRARY/PROJECT start at v1. This is because of incrementing rule inconsistencies if it starts at version 0.

If the rules define increment behavior, what's inconsistent about starting at zero? I'm definitely imagining the "long beta" libraries here that insist on calling themselves '0.2.3.4' which is a real pain right now.

5. Start Project Beta ("it makes sense, marketing told me"). It doesn't increment LIBRARY/PROJECT, so we're at `v1.4.0.0`, or optionally just `v1.3.5.2` if it makes sense under SemVar. This is confusing because `v1` covers both Alpha and Beta, two different projects. `v2` (or `vBeta`, if you will) isn't applied until after Beta is released, which is less useful.

Hmmm. Bundler handles this with suffixes in the fourth "buildid" position: 1.2.3.beta, 1.2.3.rc1, 1.2.3.rc2

It would be odd to place them into the lead position: Rails 6rc.1.2.3

In fact it seems like Rails is already following my current proposal, as Rails 6 beta was not the previous Rails 5.2.4.3, it was 6.0.0.beta or 6.0.0.rc1. This is well understood. What is inconsistent is the meaning of the second digit. Under semver it should be 6.MAJOR.MINOR.PATCH, but some read it as 6.MINOR.PATCH.buildid and still others simply view it as numbers with no real intent behind breaking vs. non-breaking changes.

And differentiating between the beta of a LIBRARY vs the actual release is a big chink in my position for LIBRARY. Not sure how to resolve that.

   Or:

6. Start Project Beta ("no really, Greek letters are cool in ad campaigns"). It increments LIBRARY/PROJECT to be consistent with being a new project, so it's now `v2.0.0.0` (or `v2.0.1.0`). When it releases, it'll be bumped to `v3.0.0.0`, which is not useful at all. (Except for those companies that really like odd and even numbers having different meanings.)

That's what I originally envisioned, however I would imagine a release candidate or beta being "v2.0.0.rc1' or 'v2.0.0.beta'. It could admittedly be sticky if it was formally 'v2.0.0.0.beta', since BUILDID is not in semver, but in several other package managers (maven, bundler, apt-get).

EDIT: No big response to your most recent post, coldnebo, I definitely agree. Going to be thinking on a formal definition that minimizes harm and maximizes the usefulness of it.

Yeah, that's really the challenge of it. But great discussion. That's really all I wanted from this proposal, was to bring out all of the conflicting assumptions that are currently standing in the way of more widespread adoption of semver.

I'd love to be able to open an issue with rails and point to a recommended solution that they might accept.

[jwdonahue]

Many have proposed four or more version fields for many reasons in the past, including platform linkage. All have died on the vine. They are not SemVer and if they are worthy, they deserve a spec of their own.

[cdrch]

While standards tend to proliferate wildly as a normal part of things, I still think it's something to try to keep under control as much as possible when reasonable. SemVer is perhaps the closest to a single (well-specified) standard present in the world of versioning software. It would be ideal to update and tweak it in minor ways that all who use SemVer can choose to adopt with a maximum of ease, so as to maximize utility for everyone.

I take contention with using "they are not SemVar" as a reason for anything - it's fairly useless, being trivially true in a literal sense (we're talking about the future of SemVar, not what it is at this precise moment) while implying something along the lines of "a four field versioning system literally cannot be the future of SemVer", which is trivially false.

[jwdonahue]

@cdrch, if you'd been hanging around here for 7 or 8 years, you'd realize that is exactly the reason these proposals do not get traction. The maintainers of the spec have made it very clear, that they do not want to make breaking changes to it. They are all protecting their vested interests in their tool chains, and in some cases, their profits. Many of them operate DevOps services of one form or another. Any breaking change, rattles more than one, billion dollar lines of business. The maintainers are busy meeting their customers needs. Hence the lack of activity here and their reluctance to discuss breaking changes. I think their current focus is on defining their version range specifications. They have an interest in that, and that's been going for about year already.

The best approach is to go fork your favorite packaging tool and add experimental features that give you the semantics you are looking for. Work with it, iron out the bugs, promote it, rinse and repeat across the majority of the packaging tools that own this spec, then write the RFC and point out how many developers are using your new experimental features. Maybe even stand-up a package feed site, at least on an experimental level, geared towards your workflows.

I certainly can't say that this proposal will never be adopted, but I think if you look around here enough, you'll find many similar proposals that have languished for many years.

[cdrch]

@jwdonahue Put that way, you make a lot of sense. Thanks for the clarifications.

Part of that I was keeping in mind, albeit through a lens of considerable optimism. I'll still cross my fingers for something along the lines of this.

[coldnebo]

@jwdonahue I can't let you off the hook that easily. :)

Any breaking change, rattles more than one, billion dollar lines of business. The maintainers are busy meeting their customers needs.

This is true. But it's also why we all suffer.

With one hand, the business wants to protect existing toolchain investments, with the other, they want to market and control the product. These are the same businesses! Shopify and Github demand of Rails, clear marketing on their architectural version (LIBRARY) so that they can plan integrations forward, even at the same time that they demand "a billion dollars" in toolchains remain stable.

I personally don't have high hopes for this proposal because it is tangled at both ends by business decisions that themselves have important business drivers. But I also think it's important to have this discussion out in the open and admit that current reality is isn't really semver, but a watered down subset of semver. In fact, the way that semver is watered down in practice may be the root of the issue:

(Part 1) Semver is simple in the increment rules about version number: X.Y.Z. Part 1 is well implemented and enforced by toolchains and has broad agreement and support.

(Part 2) Semver is hard in trying to dictate that version number be mapped to the concept of a "breaking change". There are many cases where this can only be figured out AFTER a version has been released, so the concept has been limited to the intention of making a breaking change. Part 2 is not implemented nor enforced by toolchains and has very little agreement and support across the industry. "I'll know a breaking change when I see it".

Part of the reason for my proposal is that adding LIBRARY is just an extension of the X.Y.Z math, Part 1 is easy to implement. And if it provides a semantic release valve for the larger ecosystem which is having trouble using semver in Part 2, well, so much the better. At least the industry can correctly voice their intent rather than gradually watering down semver to remove Part 2 altogether (which is what is currently happening).

In the end, it isn't me you need to convince, it's the "billions of dollars" in toolchains and the libraries and the users who are gradually watering down semver to Part 1 only. Maybe this is the best we can do? It's certainly the easiest to enforce.

[ljharb]

@coldnebo can you provide a case where you can only figure it out after the release? (one that doesn’t indicate a solvable failure of testing)

[coldnebo]

@ljharb yep. in some cases, library maintainers don't find a major integration bug during testing and alpha phases, release the library and then find out afterwards. Testability is often an issue. One I personally remember are some breaking changes with NTLM support for the Savon ruby gem, which is a very difficult thing to test without a Windows test server and AD domain.

[jwdonahue]

(Part 2) Semver is hard in trying to dictate that version number be mapped to the concept of a "breaking change". There are many cases where this can only be figured out AFTER a version has been released, so the concept has been limited to the intention of making a breaking change. Part 2 is not implemented nor enforced by toolchains and has very little agreement and support across the industry. "I'll know a breaking change when I see it".

@coldnebo See #716 for a lengthy related discussion.

As you have pointed out, the problem you are trying to address is a tooling problem. It's about transitive dependencies. In the absence of tooling that blocks updates to packages who's transitive dependencies will break your build or product, we have to abuse SemVer by getting everybody to cooperatively bump their own major version when they update one of their dependencies to the next major version.

Your proposal won't solve this problem. First, it only addresses a subset of the transitive dependency problems, second, it still requires cooperation and flawless follow-through from all parties in the dependency tree, and you have already pointed out the difficulties involved in the later. This has always been and always will be a tooling problem, whether the version string has three, four or a dozen fields in it.

If you build it, they will come.

There are many open source code bases out there that you can build on. If the demand is there and the right tool becomes available, it will be adopted. Competition is awesome motivator.

[ljharb]

@coldnebo thats called a bug - semver requires you fix/revert it in that major line, and then if you still want the change, you release it in a new major.

[coldnebo]

I think we've said all we can say about this proposal, now we're just spinning in circles.

I withdraw the proposal.