Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
x/vgo: transitive dependencies and transitive dependency updates #24500
I have a question about intent with minimum version selection. Let's say I have an app with a dependency on A which has a dependency on B. The App does not have a direct dependency on B.
App --> A (v1.1.1) --> B (v1.2.3)
Let's say B has a security bug and an update is released of v1.2.4. Dependency A has not updated to the security release of B and the latest version still depends on B@v1.2.3.
How would the App developer go about getting the security release of B prior to A updating to require it as the minimum version? Using MVS, vgo is going to install 1.2.3, right?
It either depA or App require the newer version, the newer version will be selected for App.
@kardianos thanks for clarifying. That's what I thought but I wanted to double check.
I brought this up for three reasons:
The first two are changes in expectations and experiences. It's good to be very open about them early so fewer people are tripped up by the change.
Unless there is something I'm missing, which I hope is the case.
My understanding is that, if you're App's developer, A's requirements are only taken into account the first time A is added to App (through MVS). Afterwards, App's developer will have to use
This is no different to what most package managers do, since lock files fix a specific version. You still need to be aware of updates in your dependencies (and possibly updates in their transitive closure), decide whether it is worth for you (security or not) and run a package manager command to get an updated version.
Your original statement is true for new apps (say App2). If you create App2 and adds A as dependency, MVS will include the B's version that A was last tested with by A's developer, which is the one not including the latest (security or not) fixes. But (if you think of it) this doesn't change the picture: App2's developer still has to continuously monitor fixes of its (transitive) dependencies for correct maintenance of their application; B's security fix might be released the day after A is added to App2, so you either monitor this or run the risk.
If you want to update all your dependencies to the latest version no matter what, you can still run a
@rasky I'm not sure we're on the same page for how most package managers work. Let me explain where I'm coming from and how it works.
The data we collected suggest that Go developers are most familiar with npm, pip, gems, maven, bower, bundler, easy install, cpan, composer, and cargo (first 10 in the list anyway). When I look at most package managers I'll try and do so in terms of what Go users expect from their experiences.
Most of these have the concept of a manifest and a lock file. The lock file keeps the current revision, and often some more details, that are in use for the complete dependency tree (direct and transitive).
The manifest file holds details on direct dependencies of an application and a range is typically included rather than a specific version to use. The range allows the package managers resolver to look for the maximum compatible version based on ranges (as opposed to the vgo minimum compatible version).
Using a tools like npm (and many of the others on that list), an app developer can run to update (withing the realm of of compatible range). For example,
Because it looks at the maximum compatible version and ranges are often used, security updates like the one in the original issue, are picked up with no extra work from the app developer.
We did a pretty through analysis of the package managers people have the most experience with. The features in dep (and to a slightly lesser extent in glide), when it comes to handling updates, are the same patterns used in the popular package managers for other languages. You can find a roll-up of details here.
If you'd like to view the details of the survey, including that a majoring of go devs want to use ranges, you can find them here
I tried to answer to the question you wrote in the first post, which is how to handle a specific single version upgrade to handle a security fix. I tried to depict different scenarios and compare them.
You're now making an example where you want to upgrade all dependencies to the latest compatible release (
If your point is just that vgo is different from the others, then yes it is. If you want to have a constructive discussion, you need to explain a specific scenario and a specific use case, and we can discuss about differences, pros and cons.
For example, if I import Kubernetes client-go it has a bunch of dependencies of its own. And, as I update to newer version of client-go the dependencies it has changes. The managing of those and their versions.
The app developer can run "go get -u" to update everything to latest minor version, or (eventually) "go get -p" to update everything to latest patch version. The requirement simplification algorithm (Algorithm R in the mvs post) means that in your situation the App's go.mod will show B v1.2.4 explicitly. If, later, A v1.1.2 adds a requirement on B v1.2.4, then when App updates to A v1.1.2 the mention of B v1.2.4 in go.mod will become redundant and will be dropped automatically.
Yes and no.
In your example, at the point where the App picks up A for the very first time and picks v1.1.1, then yes Cargo/Dep/others will see that A v1.1.1 asks for B v1.2.3 and automatically pick up B v1.2.4 instead if it exists. So yes, vgo would pick up B v1.2.3 in this situation and that's a change in user expectation. This is absolutely by design. The minute before v1.2.4 comes out, vgo picks up B v1.2.3. The minute after v1.2.4 comes out, vgo still picks up B v1.2.3. No one has asked for v1.2.4. It may be a terrible release that's going to be yanked in an hour. Until someone says explicitly "I want to do an update operation", with the implied "I'm ready for things to break and to deal with that", I don't believe the build system has any business pulling in brand new releases.
Note that if even when Cargo/Dep picks v1.2.4, from that point on there's a lock file keeping the App at B v1.2.4, even if v1.2.5 exists, until the next version adjustment. Even in Cargo/Dep, once you do the initial pull, you still don't get future upgrades unless you ask for them. It seems like the Cargo/Dep developers believe the same thing I do, in at least some contexts. That belief is just not as consistently applied in Cargo/Dep as in vgo. (I'll also note that Sam Boyer has talked in the past about making Dep treat A's suggestion of B v1.2.3 as a "preferred version" to avoid pulling in v1.2.4 automatically.)
It is certainly true that some users may expect that adding A will get the latest version of A and then the latest version of all its dependencies automatically, so that A's author can be lax about updating A's own dependencies. I think I've been very up front that instead vgo pulls the latest version of A and then the versions of A's dependencies that A was actually built and presumably tested with, or as close as possible to that ideal. This is of course in direct conflict with the "I want the latest of everything" argument. But the people who want the latest of everything can use "go get -u A" instead of "go get A". In contrast, there's no way in Cargo/Dep to say "get me what A actually used".
The only extra work is typing "go get -u" when you want to override the default and bring in the absolute newest of a set of dependencies (and are ready to test and debug the result). Or "go get -p" if you think that's meaningful (we'll see whether it is).
Yes there's extra work, but not much. A new tool that types -u for you doesn't seem like it will be worthwhile.
From this, it looks to me like if you had a system that forced newest dependencies always, like Cargo/Dep, then you'd be alienating the 35% of respondents who only want that sometimes (implicitly, not all the time). Better to give developers the option instead.
I think the survey question is a bit misleading, for two reasons. First, I assume you didn't also ask "It's important to me that I control when new dependencies are pulled in." Second, the question says "latest compatible". Obviously everyone would agree with that in the abstract. The problem is that
Minimal version selection attempts to strike a balance between these two competing concerns, so that updating to latest of everything is easy, but developers get more control over exactly when that happens.
Agreed. I've tried to be very up front about this distinction from other systems, and I will continue to be.
I'm going to close this issue since it seems like your initial question has been answered.
@rsc First, thanks for clarifying. I've gotten contradictory information which is why I asked the question in the first place.
Second, I get the impression we're doing a bit of reinventing the wheel going on.
As someone who has worked on dependency managers I know that people want to store the rules, per dependency, in configuration. Requests for features quickly show up around that. Have you talked with people, other than those on dep, about their experiences building a package manager to know where they've gone and why?
Third, It's a little disheartening to know that effort was put into survey the Go users on dependency management to ascertain their needs and you've not read it. If you had you would have known what was asked and the results. Building a dependency manager without knowing what the community has stated as their needs should cause the community some caution because it gives the impression you're not listening to them. Even when someone on the Go team at Google was involved in conducting the survey.
One thing I don't often read is comments on closed GitHub issues - there are just too many mails from GitHub - so I hadn't seen this response. It was pointed out to me earlier today that some people are citing this response as evidence that I ignored all the prior work done by the dependency management group and committee, which is unfortunate.
My only comment on this thread, above, critiques the survey question wording and the conclusions you drew from the response.
There is absolutely no support for your claim that I didn't read the survey.
To set the record straight, in fact I did read the survey and all the docs generated from the working group, and I talked through the company survey results with Steve Francia multiple times, all in the winter of 2016-2017.