-
Notifications
You must be signed in to change notification settings - Fork 691
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Transitive dependencies are part of public API and thus should increment version accordingly #341
Comments
This actually isn't specific to Java, but applies to any system that needs to resolve dependencies to a single version. (Which I think is actually most package managers, with the notable exception of NPM [which will instead wantonly break the build by installing incompatible versions of a dependency even when they are part of the API.]) |
I agree with the MAJOR change rule being generally applicable to most package dependency systems. This new rule leads to a new kind of necessary "MAJOR version increment hell" where a commonly referenced project makes a breaking API change and requires all upstream dependencies libraries / systems to make changes and subsequently increment MAJOR as well. In the cases of MINOR and PATCH, many dependency systems let you specify version ranges and in most cases (>= A.0.0) AND (< A+1.0.0) work fine. So I don't know if there is a general need to increment your MINOR & PATCH every time a transitive dependency posted a new version. That would simplify the amendment down to something like: Transitive dependencies are part of the public API thus the version of the user library SHOULD be incremented when a direct dependency makes an incompatible API change:
This rule applies recursively for all transitive dependencies. |
This is a "what is being versioned?" problem. If you are shipping a dependency in your package, and that changes, your package version has to be bumped, but is it a patch, minor or major change? If it's the kind of package that works perfectly fine in side by side (SxS) installations and review/test shows little risk of breaking your customer's code, then it might just be a patch bump. It always comes down to what impact does this have on your customers? If it does not support SxS, then your customers must take the update to use your product, but this could break other products on their system that also rely on that package, so you need a matching minor or major version bump. If you do not ship the dependency in your package and it changes without any corresponding changes in your code, no bump is needed. If you modify anything in your package, even a manifest, you have to bump something. Again, if you force your customers to take a change that could impact any other software installed on their system, you must bump either the minor or major version, depending on the level of changes made in that dependency. I think the spec covers all of this, it's just that the full ramifications of it aren't always well understood. So I think I just talked myself into supporting some kind of change in the FAQ to cover these aspects, I don't think it would be Java specific at all. |
@jwdonahue For example, let's take a look at
Looking at this suffix
So switching 0.16.5 to 0.16.5a will cause major incompatibility in user's code despite that these versions have the same code except the transitive dependency. That's why I think that this innocent looking suffix |
I see your point, can you give us any examples where the major version >=1 ? I fear it may be unfortunate that you chose an example where breaking changes are in fact to be expected from any minor change, because the major version is zero. I should also point out that version labels you use as examples, do not adhere to the SemVer spec. |
This is a real example we have seen in our project. I can't remember a real example with semver versions. Indeed, the version labels are not SemVer. However, we might replace the version labels accordingly. For |
Ok, I think the spec already covers these cases. If you introduce a breaking change, you bump the major version. Should the FAQ cover every scenario that is a breaking change? Probably not, is Java worthy of a special mention? Perhaps. Please do one or more of the following:
|
@Primetalk, apologies for ignoring this for so long. I appreciate that you have expended the effort to issue a PR and I think that should probably be merged. Unfortunately, @haacked has been very busy over the past year and might not get to it for a while. I have created issue #468 in order to step back and take a holistic look at the whole API/Package/Dependencies issue. I've come to the conclusion that if we work this problem piecemeal, we could make it worse, but some additional FAQ's such as yours might help. I also suspect that the spec can't really be fixed without a 3.0.0 version of the spec. |
ImgBot |
1 similar comment
ImgBot |
On the original topic:
This is a way to go only if a new version of A stops working with old version of d. |
|
TL;DR: It's the job of your package manager/runtime to guarantee that the semver constraints for every package are satisfied. Consider the following dependency graph. Application:
Framework:
Changing the major version to In general I disagree that transitive dependencies are part of the public API. The public API is what you define it is. If you you say that your library function returns an iterable then it is not breaking if you change your internal implementation (provided by a dependency) from a LinkedList to a TreeSet. The consumer might find it important to know how many 3rd party dependencies exist but it's entirely up to the library maintainer to define this as part of the public API or not. Edit: |
In your example, both app and framework depend on library. And they require different versions. Strictly speaking this configuration should not be allowed at all. Let's consider the following changes in There are the following possible failures:
Public API of framework doesn't change. In both versions method Tests of framework do not cover this case, because the results of lib are not used directly and instead returned to the application. Application, in it's turn, expects that Framework return correct results - In this case if the actually used version is lib 1.0 application will fail.
There is no good decision that could be made based on these erroneous versions. On the other hand, if we explicitly promote this difference in the version of framework, then it'll be obvious that application has to be upgraded. |
@Primetalk Why? I made that argument under the premise that your package manager (or runtime actually) should ensure that the semver constraints for each module is fulfilled. I guess some runtimes don't do this? There's no useful argument to be had if you argue under the assumptions that runtimes are not responsible for semver constraints while I think they should. At that point we're arguing about implementation not specification. Package managers and the node runtime, however, show that you can change transitive dependencies in a patch without violating semver for consumers of your package. |
The question is not if some package managers can resolve some transitive dependency updates, the questions are:
The answer to 1) is very clearly "No". An easy example is library with a stateful singleton as only export. Either there will be a conflict in interfaces or the singleton nature will be violated if package manager/runtime provide differing instances. This leaves us with 2), which really is a question about the definition of "(public) API" in the SemVer spec. The usual understanding in the developer community seems to be "the objects my package exports, their names, their interfaces and possibly their path relative to the package root". This is enforced by the answer to the FAQ "What should I do if I update my own dependencies without changing the public API?", but strictly speaking this FAQ doesn't actually make any claims about what "public API" actually means. It just asserts that if your public API is unchanged, SemVer does not mandate a MAJOR version bump. I agree with @Primetalk that dependencies can be part of the public API of a piece of software and that the SemVer spec and the FAQ should be updated to improve verbiage and clarity around this topic. At the very least there should be a note that for many authors of libraries, what is versioned does not include dependencies. My personal benchmark for "Is this a MAJOR change?" has always been "Could there be a consumer that uses my code as documented/intended who will need to change code (as opposed to package manager configuration) to consume my new package?". |
But that is what you define:
SemVer does not define it for you. It sounds like this question is about what is implied by a certain language/runtime/ecosystem when defining a public API and not about transitive dependencies in general.
Emphasis on can. If it is not always the case SemVer should not mandate a bump. |
But this does not define what a public API is. Worse, the very next sentence is (emphasis by me)
Now combine this with (emphasis by me)
from https://en.wikipedia.org/wiki/Application_programming_interface and I am hard-pressed to find a way how API can not include the methods and objects of your dependencies you use.
But the current spec (or rather, the FAQ that are in the same .md file) does not even hint at can.
In RFC2119 terms thats a SHOULD NOT at best, but it could easily interpreted as a MUST NOT. Neither @Primetalk nor I want to mandate a bump. The PR @Primetalk opened uses SHOULD which is a recommendation, but not a mandate. |
The semantics of your public API are part of it. If a dependency version bump changes that, then semver applies. If it does not, then there's no observable change, so there's nothing to bump (besides what you'd normally increment for publishing). |
I gave you an example for this:
I could conjure up plenty of examples for this. The important part is if a dependency is an implementation detail or not. I'm hard-pressed to accept that every dependency is part of your public API. If it's not public then it's not public. There's nothing special about dependencies. Even the mere existence of a dependency is an implementation detail. It doesn't matter if you have a dependency tree that is 3 levels deep or if every dependency is inlined. This is probably the best advice for evaluating transitive dependency impact on SemVer: "If every dependency would be inlined (removing the notion of dependecies) would my public API change?" |
You just shifted your dependency from e.g. Guava to the JVM version (Iterable was introduced in 1.5) without providing any further argument why runtime dependencies of your code should not be included in what is versioned. Iterable in Java 1.5 does not have a |
That is true whether you have transitive deps or not - which is why “whether a transitive dep change affects your version” is dependent on the actual change and not on the mere fact that the transitive dep changed. |
I do not disagree with that, but the SemVer spec* does, because it says that
The sentences after that try to "soften" it a bit, but this statement implies a distinction between "public API" (which is versioned under SemVer) and dependencies. |
I find it very clear; the thing that’s part of your api is your usage of your deps, not the deps themselves. Do you really believe that the authors who under report a breaking change are doing so because this verbose spec has multiple possible intepretations? If so, have you tried a PR with your suggested wording? |
If it was clear to everyone, this whole thread would not exist ;)
Even that statement is too absolute. Singletons, especially stateful ones, are a good example where simply bumping the version of a dependency without any other code change may be a breaking change, e.g. in node where the singleton nature of a module depends on the absolute path and the resolution of name to path depends on the path of call-site. Nested dependencies compound this problem.
Seen this more than once and at least one developer pointed me to that very FAQ to justify his patch level release when doing a major upgrade of a dependency shared. That is actually what lead me to find this discussion here.
I have not. @Primetalk has however and the slow progress on that gives me little hope that a PR of my own would have much chance of being accepted at this time. I will think about it some more though. |
@eps1lon This issue is about that on at least JVM platform there is a leak of transitive dependencies via unchanged public API. (The same problem exists on all platforms where only one version of a dependency can be used by the same runtime.) |
That is your opinion. You talk like this is some objective truth by tying SemVer to Java. I don't agree that this is the real problem. The real problem is that you changed your public API. How you did this is by not curating your transitive dependencies. SemVer should not dictate how changes are made but what changes are made. It is the responsibility of library authors to carefully define your public API.
Sounds like they already are. SemVer is not runtime specific and should therefore make no claims about how the runtime should behave. |
Ok, great.
This is what the issue about. It describes that transitive dependencies are part of public API. And recommends to bump major version whenever transitive dependencies change major version. |
Seriously? Replace Guava with [wherever you get LinkedList from] and JVM with [wherever you get Iterable from]. Happy now? Again, the point is not that you can conceive some scenario where bumping a dependency of your library has no impact on consumers, but that there are many scenarios where it does and where the impact is "consumers code breaks". |
If they’re a part of your public API, then it’s a major bump when your public API has a breaking change. Whether the mechanism for that is that you bumped an improperly encapsulated dep or that you edited the code, it’s still not directly related to your transitive deps. |
That is precisely the wrong model for most languages/runtimes. Inlined code is isolated to its scope, dependencies are shared and in most runtimes that means all the code in the runtime needs to agree upon how to interact with the dependency and how the dependency itself may interact with the runtime environment. |
Could you name the languages where this is not the case?
Not true for javascript. The node runtime has solved this problem with |
Languages for which import statements are not the same as inline code (partial list): JavaScript, Python, Ruby, PHP, Java, C, C#
Which only solves the problem for stateless modules. The moment your module does
Webpack just requires the app developer to resolve any conflicts as it flattens the dependency tree. Only one of the version will make it into the final bundle (unless you create intermediate bundles that hide the shared dependency, in which case you get the same singleton issue you have with node).
Same issue as regular node, if a module is supposed to be a stateful singleton, that assumption is only true as long all code agrees on the version. |
Alright I guess I made my peace. You are very creative with constructing specific examples (now stateful modules) that break the public API but have in general nothing to do with being a dependency. Making transitive dependencies a MUST in the public API would be a breaking change for the SemVer spec. If this would be accepted I'm pretty sure any stateless library will not use that version of the spec. |
I am not asking for MUST though, I am asking for MAY. https://github.com/tkissing-work/semver-dependencies-are-api is a very simple example of how even in node a the problem is not solved for all cases. I feel like a broken record, but I will say it again: |
The spec talks mostly about versioning API's, not packages, art or cars, yet the semantics can be applied to almost anything that requires versioning, if you are at least reasonably aware of exactly what the version number applies to. The word Package appears in the specification only once, but ten more times elsewhere in the document. SemVer never set out to solve the diamond point, or any other dependency problem. It's not a tool, it's just a description of one way to version API's. Unfortunately, it also mentions packages. Well the only players in the game, that could successfully implement, support and promote SemVer, were the packaging tool owners, how could it possibly ignore them? Now it's the package tool owners who own the spec. Are any of them anxious to see the spec define things like how to version something with transitive dependencies? Probably not. Each of them owns one of the dominant packaging tools for their particular language or framework. They handle dependencies differently. |
Thanks everyone for contributions, you're amazing 🎆 Did you find any consensus? |
Closing as staled, feel free to re-open or create a new issue 👻 |
What will be the destiny of the PR - #414? |
Some other code is not responsibility of A's author. A library author's responsibility to indicate if change in a library breaks or does not break A's consumer. |
IMO, the main problem with this proposition is the difficulty of applying it in the real life. If a transitive dependency uses SemVer then sure. I limit major version anyway, and I know that when they release a new major they are breaking something, so does me. But if a transitive dependency doesn't use SemVer I cannot be sure whether their new version is a breaking change or not. It's not my code, I just use several features of it. The proposition that could be really discussed is |
No, if you use a dependency that doesn’t use semver, you’re responsible for pinning it so that consumers aren’t broken. every dependency of yours is part of your public api. |
Your users could ask the same question: Why should they have to read your changelog and all of your dependencies changelogs in order to tell whether your new version is a breaking change. Keep in mind if some other library has conflicting requirements it would even be a problem for your users who don't use the transitive dependency directly. |
What should I do if I update my own dependencies without changing the public API?
For Java the best judgment can be more precise.
Let C be user's project, A be our library and d be a transitive dependency. And let's say that we want to use a newer version of d - d' - in our new version of A'. In Java application with the single class path there are two options: either transitive dependency jar be available on the user's classpath in the version d', or it'll be available in the previous version d (ceteris paribus). In the former case other modules/libraries that were expecting older version d would suffer from incompatibility, in the latter case our library would suffer. The degree of change is exactly equal to the degree of version change of d (MAJOR, MINOR, PATCH). Thus d is effectively part of the public API of our library A.
Of course, we suppose that d follows semver recommendations.
Proposal: Amend this FAQ section and add a special notice for Java (as an important domain for semver).
For Java (and similar environments) transitive dependencies are part of the public API and thus the version of the user library SHOULD be incremented to the same degree:
If a few dependencies are being changed, only the highest degree is required to be incremented.
The text was updated successfully, but these errors were encountered: