Definition of backwards compatibility

[jbrann]

semver specifies the patch version should be updated for "backwards-compatible bug fixes." What is being considered for backwards compatibility? This could be:

1. the documented behaviour of the system
2. the actual behaviour of the system

there are many cases where actual behaviour is not documented (because it's too fine-grained or low-level) or because the system has a bug.

Both options lead to the possibility of undesired outcomes:

1. if changes that do not affect documented behaviour are not major, then clients that have implemented code that uses incompletely documented features, or works around bugs will probably be broken by a patch release.
2. if changes to the behaviour of a system result in a MAJOR version changes, then every bug fix release must be a major version change.

What practices are recommended for avoiding outcome 1 while minimizing the upheaval inherent in outcome 2?

[pylang]

Similarly, I have an issue where the API of one of my dependencies has changed (one of the attributes was deprecated and renamed). To fix the bug, I will have to either do nothing (and pin the dependency version) or refactor my library to reflect the recent changes. In general, refactoring any deprecated attributes means an older package cannot use those new dependency attributes, making them "backward-incompatible". Yet it seems excessive to bump a major release version each time a dependency decides to change its API slightly. How can semver resolve "backwards incompatible" patch releases caused by API changes in dependencies?

[crazedsanity]

I think it all depends upon how much it affects your library. If that API change had zero net effect on your library, its probably worthy of a patch release; if that change caused a massive change and makes the change to your library break anything that upgrades to it, then a major version bump sounds right. Just my two (or three) cents.

[mparker]

Since "backward compatibility" (and breaking it) is a fundamental concept to semver, it would be nice to have at least a non-normative definition of it in the spec. It might be that it means something different depending on the semantics of the application that's following it, and perhaps that could be mentioned.

[jwdonahue]

@jbrann, if you require further clarification please speak up, otherwise, please close this issue at your earliest possible convenience.

[jbrann]

This has been open for a while and (together with the linked comments) appears to highlight some questions around fundamental semantics. If we intend to state in a version number that a consumer should be safe using the new version (i.e. that the new version is backwards compatible with a previous (by which I mean available earlier in time) version by a pattern matching rule on the version) then that statement is PROBABLY limited to the expected behavior of both versions.

Expected behaviour is something like "the superset of documented behaviour that is actually observed and might be reasonably inferred from the documented behavior."

Bug fixes that break actual behavior (100% do) but not expected behaviour are legitimately minor and should be viewed as "backward compatible," even though they break clients with work-arounds.

I'm afraid this language is just full of the kind of grey areas that I would want to remove, but I can't do better.

I'm willing to close the discussion if there are no more contributions

[jwdonahue]

@jbrann, apologies, I may have been on auto-pilot by the time I got to this thread. Thanks for the reasoned reply. I do agree the spec could be more informative than it is, in some places, but I am hard pressed to come up with adequate language to replace it. You are certainly welcome to give it a try, just issue a PR.

Sometimes, what the spec and FAQ don't say, is as important as what they do say. There are obvious edge cases as you point out, where the strictest adherence to what they do say, could lead to version inflation. I think we all have a strong desire to produce, and purvey the appearance, of stability and predictability in our products. Obviously, bumping the major version number with every little change will not lend to that appearance, nor will breaking your customers systems with every little patch or minor bump. Providing guidance in this area is complex and can't really be covered in one or two FAQ paragraphs and probably shouldn't be tackled head-on in the spec, which must leave some room for subjective interpretation or it risks becoming overly complex. It would just be impossible to cover every aspect anyway.

If your interface is weakly documented for its level of complexity, you are more likely to discover that some of your customers have taken dependencies on behavior you did not intend for them to assume. This happens even for well documented systems actually, it has always amazed me how end-users can find ever more creative ways to "abuse" my products. If a bug fix removes what some customers consider a feature, and you don't bump the major version, you will piss them off. It's a matter of your own policies exactly how much pissing off you are willing to do.

Performance and footprint are common assumed features, whether or not they have been advertised. Bug fixes and refactors can have measurable impacts on performance and footprint, both positive and negative. You might not think that a speed-up or size reduction would be a bad thing, but they can both cause dependent systems to break. Should you have bumped the major version for that refactor or bug fix? That depends on your relationship with your customers, it's probably a bug in their code, but if their code is used in millions of products and end-users took your bug-fix, well... Perhaps nothing broke, but one of them sues you for not informing them of the potential speed-up because one of their competitors is suddenly beating them on the trading floor? Do you have a small number of direct customers who use your product in products that ship to millions of people or just a few dozen?

I don't have time to go on with other examples, but I can think of many and I am sure that the set of possible examples I don't know, is much larger than the set that I do know of. My point is, that every application of SemVer is different in ways too numerous for anyone to enumerate, let alone weave a coherent story that all of them used some fixed mapping from the spec to their application. Apply your best judgement and be as transparent as possible regarding your policies with regard to your application of SemVer.

[jwdonahue]

@jbrann, well it's been five days and there have been no further comments, it would help if you could close this issue, so we can organize around the issues that are going to take our valuable attention, unless you intend to issue a PR?