Feed priority when same package is available in multiple feeds

[Sam13]

Applies to all versions of NuGet.exe (v3.3.0 to 4.3.0 RC).

When a package with same id and version exists in different feeds I always assumed that the package source priority is taken from the order of package sources in the NuGet.config.

But that seems not to be true. As far as I understand the code in https://github.com/NuGet/NuGet.Client/blob/dev/src/NuGet.Core/NuGet.PackageManagement/PackageDownloader.cs - GetDownloadResourceResultAsync()
all feeds are queried and the one with the shortest response time wins. Is this correct?

If so - what is the proposed solution to the following problem:
I have a private feed which is listed before the official NuGet.org feed in my NuGet.config. In my private NuGet feed I have a package with id A and version 1.2.3. Now someone else creates another package A, version 1.2.3 and uploads that to the NuGet.org feed. This package has different content than my private package.
With the current behavior of the NuGet.exe it's random which package A is downloaded.

Sure I could try to use a prefix for my private package ids that nobody else may use, but who can guarantee that this id is never used on NuGet.org in future?

By having a priority for the package sources this problem can be solved. Any other ideas?

Thanks in advance

[emgarten]

all feeds are queried and the one with the shortest response time wins. Is this correct?

That is correct, there is no longer a priority for sources, the fastest one wins.

Packages are expected to be unique on the id and version, scenarios where feeds contain different variations of the same id/version are NOT supported.

Sure I could try to use a prefix for my private package ids that nobody else may use, but who can guarantee that this id is never used on NuGet.org in future?

We've looked at addressing this in the past, but currently there is no way to avoid collisions with other servers beyond using your private feeds.

[Sam13]

Thanks @emgarten for the clarification.

So what about introducing an OPTIONAL priority for feeds to solve that problem?

NuGet.config might look like this (assumed that the smallest number is the highest priority):

<packageSources>
	<!-- Internal package source comes before NuGet.org proxy -->	
	<add key="my_private_feed" value="http://internal-server:8081/repository/private_feed1/" priority="1" />
	<add key="cache_nugetorg" value="https://www.nuget.org/api/v2/" priority="2" />
</packageSources> 

The PackageDownloader needs to be adapted that he waits for the download with the highest priority if package sources have different priorities otherwise fastest wins (as it is now)...

[emgarten]

@Sam13 thanks for the suggestion, adding a priority would need to be respected for all places where multiple sources are used including for querying for package metadata to ensure that the right package was found. At this time I don't see this fitting into the current client plans which are moving away from ordered sources. My guidance on this is to avoid having different packages with the same id/version, or to reconcile these issues before hand by using a single feed where this has already been resolved.

[johntiger1]

Thanks for this thread, saved me some time. Although personally I still think that more features (i.e. priority on package sources) is better.

[AndrewDorn]

"I don't see this fitting into the current client plans which are moving away from ordered sources." Doesn't make sense. What DOES make sense is that this is being done to discourage having more than one feed, since that's the only way to avoid NuGet making guesses as to what you want when collision happens. I fail to see how specifying a specific feed causes any deviation from the current path... It's literally just limiting the feeds contacted.

I imagine most, if not all issues that people have due to this could be solved with an optional 'Source'/'Sources' parameter on package references/an optional source parameter when requesting a package (maybe default to filling it in with whichever source the package was initially installed from when it comes to references, with the option to remove it if you want to use any source). Cannot think of any reason not to support this, either.

Not sure how the local caches are currently structured, but might need a rework to track which source cached packages were downloaded from.

This really does seem like a pretty fundamental issue; if you have the option to have multiple sources, you need to have some reliable way of dealing with conflicts.

[beatcracker]

@iBeizsley
Cannot think of any reason not to support this, either.
@emgarten
My guidance on this is to avoid having different packages with the same id/version

Please be aware that this non-determinism is now being abused for supply chain attacks 😞:

• Researcher hacks over 35 tech firms in novel supply chain attack
• 3 Ways to Mitigate Risk When Using Private Package Feeds

[hypervtechnics]

@beatcracker I think you meant this link here: https://azure.microsoft.com/en-us/resources/3-ways-to-mitigate-risk-using-private-package-feeds/ for the second item 😄

I am honestly confused why this "fastest"-thingy behaviour is actually used for an ecosystem like NuGet...

[beatcracker]

@hypervtechnics Right, mornings are hard on me. Fixed, thanks!

[alfredmyers]

@beatcracker I think you meant this link here: https://azure.microsoft.com/en-us/resources/3-ways-to-mitigate-risk-using-private-package-feeds/ for the second item 😄

I am honestly confused why this "fastest"-thingy behaviour is actually used for an ecosystem like NuGet...

I work at a place that blocks all direct access to NuGet.org due to, as we can now see, justified security concerns and instead uses a 3rd party product as a proxy for package sources. As a consumer of such package sources, I can tell you the people in charge of keeping those servers running seem to have a hard time doing so because I'm constantly receiving e-mails about scheduled & unscheduled downtime due to maintainance, rebooting, etc.. That may be one of the reasons for them having several different instances of the proxy which materialize as having several internal package sources being configured.
When the operation is going to succeed (due to one of the servers being able to serve it) the "fastest"-thingy ends up helping as the response time is bound to the fastest server.
On the other hand, in case of failure, the response is only displayed after the last server responded with a failure status/message. In this case, the response time is bound to the slowest server.

Turing off "fastest"-thingy is not a solution given that if for any reason the intended package source isn`t able to service the resquest (downtime for instance) and other package sources are configured and enabled, one-by-one, each one of them will be given the chance to service the request until the first one succeeds - or all of them fail.