Feed priority when same package is available in multiple feeds

[Sam13]

Applies to all versions of NuGet.exe (v3.3.0 to 4.3.0 RC).

When a package with same id and version exists in different feeds I always assumed that the package source priority is taken from the order of package sources in the NuGet.config.

But that seems not to be true. As far as I understand the code in https://github.com/NuGet/NuGet.Client/blob/dev/src/NuGet.Core/NuGet.PackageManagement/PackageDownloader.cs - GetDownloadResourceResultAsync()
all feeds are queried and the one with the shortest response time wins. Is this correct?

If so - what is the proposed solution to the following problem:
I have a private feed which is listed before the official NuGet.org feed in my NuGet.config. In my private NuGet feed I have a package with id A and version 1.2.3. Now someone else creates another package A, version 1.2.3 and uploads that to the NuGet.org feed. This package has different content than my private package.
With the current behavior of the NuGet.exe it's random which package A is downloaded.

Sure I could try to use a prefix for my private package ids that nobody else may use, but who can guarantee that this id is never used on NuGet.org in future?

By having a priority for the package sources this problem can be solved. Any other ideas?

Thanks in advance

[emgarten]

all feeds are queried and the one with the shortest response time wins. Is this correct?

That is correct, there is no longer a priority for sources, the fastest one wins.

Packages are expected to be unique on the id and version, scenarios where feeds contain different variations of the same id/version are NOT supported.

Sure I could try to use a prefix for my private package ids that nobody else may use, but who can guarantee that this id is never used on NuGet.org in future?

We've looked at addressing this in the past, but currently there is no way to avoid collisions with other servers beyond using your private feeds.

[Sam13]

Thanks @emgarten for the clarification.

So what about introducing an OPTIONAL priority for feeds to solve that problem?

NuGet.config might look like this (assumed that the smallest number is the highest priority):

<packageSources>
	<!-- Internal package source comes before NuGet.org proxy -->	
	<add key="my_private_feed" value="http://internal-server:8081/repository/private_feed1/" priority="1" />
	<add key="cache_nugetorg" value="https://www.nuget.org/api/v2/" priority="2" />
</packageSources> 

The PackageDownloader needs to be adapted that he waits for the download with the highest priority if package sources have different priorities otherwise fastest wins (as it is now)...

[emgarten]

@Sam13 thanks for the suggestion, adding a priority would need to be respected for all places where multiple sources are used including for querying for package metadata to ensure that the right package was found. At this time I don't see this fitting into the current client plans which are moving away from ordered sources. My guidance on this is to avoid having different packages with the same id/version, or to reconcile these issues before hand by using a single feed where this has already been resolved.

[johntiger1]

Thanks for this thread, saved me some time. Although personally I still think that more features (i.e. priority on package sources) is better.

[AndrewDorn]

"I don't see this fitting into the current client plans which are moving away from ordered sources." Doesn't make sense. What DOES make sense is that this is being done to discourage having more than one feed, since that's the only way to avoid NuGet making guesses as to what you want when collision happens. I fail to see how specifying a specific feed causes any deviation from the current path... It's literally just limiting the feeds contacted.

[ayesaac]

I imagine most, if not all issues that people have due to this could be solved with an optional 'Source'/'Sources' parameter on package references/an optional source parameter when requesting a package (maybe default to filling it in with whichever source the package was initially installed from when it comes to references, with the option to remove it if you want to use any source). Cannot think of any reason not to support this, either.

Not sure how the local caches are currently structured, but might need a rework to track which source cached packages were downloaded from.

This really does seem like a pretty fundamental issue; if you have the option to have multiple sources, you need to have some reliable way of dealing with conflicts.

[beatcracker]

@iBeizsley
Cannot think of any reason not to support this, either.
@emgarten
My guidance on this is to avoid having different packages with the same id/version

Please be aware that this non-determinism is now being abused for supply chain attacks 😞:

• Researcher hacks over 35 tech firms in novel supply chain attack
• 3 Ways to Mitigate Risk When Using Private Package Feeds

[hypervtechnics]

@beatcracker I think you meant this link here: https://azure.microsoft.com/en-us/resources/3-ways-to-mitigate-risk-using-private-package-feeds/ for the second item 😄

I am honestly confused why this "fastest"-thingy behaviour is actually used for an ecosystem like NuGet...

[beatcracker]

@hypervtechnics Right, mornings are hard on me. Fixed, thanks!

[alfredmyers]

@beatcracker I think you meant this link here: https://azure.microsoft.com/en-us/resources/3-ways-to-mitigate-risk-using-private-package-feeds/ for the second item 😄

I am honestly confused why this "fastest"-thingy behaviour is actually used for an ecosystem like NuGet...

I work at a place that blocks all direct access to NuGet.org due to, as we can now see, justified security concerns and instead uses a 3rd party product as a proxy for package sources. As a consumer of such package sources, I can tell you the people in charge of keeping those servers running seem to have a hard time doing so because I'm constantly receiving e-mails about scheduled & unscheduled downtime due to maintainance, rebooting, etc.. That may be one of the reasons for them having several different instances of the proxy which materialize as having several internal package sources being configured.
When the operation is going to succeed (due to one of the servers being able to serve it) the "fastest"-thingy ends up helping as the response time is bound to the fastest server.
On the other hand, in case of failure, the response is only displayed after the last server responded with a failure status/message. In this case, the response time is bound to the slowest server.

Turing off "fastest"-thingy is not a solution given that if for any reason the intended package source isn`t able to service the resquest (downtime for instance) and other package sources are configured and enabled, one-by-one, each one of them will be given the chance to service the request until the first one succeeds - or all of them fail.

[aceoft]

Just started doing our own private nuget feed to try to do dependencies the "right" way and this is the very first thing I run into. I used a fairly standard name (thinking it was private/unique to my organization only), and within minutes, VS is trying to "upgrade" it to someone else's code.

Anyone in the world can publish a package with the same name I'm using, and Visual Studio will happily suggest that I replace my package with theirs due to the version number being higher. This is a MAJOR security flaw, and it has been documented clearly and concisely above.

The response seems to be, "just never screw up and do it accidentally, guy." Really? And I have to train my whole team to never do it by accident as well? Or pay for some other tool that will merge feeds and disallow usage of nuget.org entirely? I cannot even begin to try to enforce that across my org.

So instead I have to come up with some very unique name and hope that nobody ever bothers to exploit it if they feel like it. Cool, thanks.

What is everyone else doing to work around this?

[ayesaac]

@aceoft inverse of my suggestion above; package name pattern matching for specific sources

https://devblogs.microsoft.com/nuget/introducing-package-source-mapping/

Basically, nuget.conf in each of your solutions, use a prefix on all your internal packages, match that prefix on your internal feed.

[zivkan]

Additionally, it's possible to reserve ID prefixes on nuget.org. It does mean that all your company internal packages will need to use the same prefix, but with the combination of package source mapping + reserved prefix, there's good risk mitigation against this type of issues.

[aceoft]

This looks promising, thank you so much for taking the time to reply!

[raducucu82]

all feeds are queried and the one with the shortest response time wins. Is this correct?

That is correct, there is no longer a priority for sources, the fastest one wins.

Is this still valid for .NET SDK > 6.0.100-rc1? Was trying to debug and issue and this will explain it 😄

[zivkan]

I'm not sure when feed priority was removed (honestly, I didn't even know it was ever a feature, but the first comment from an ex-team member implies that it was), there certainly has never been a feed priority since PackageReference was introduced (NuGet 3.0, .NET Core 1.0, including all the project.json prototypes before 1.0).

Also note that now Package Source Mapping is a feature that can instruct NuGet to only search specific source(s) for certain package ids or prefixes. Keep in mind, NuGet will not download a package, even when package source mapping is enabled, when the same package id-version is already in the global packages folder.

I still strongly recommend everyone use either CompanyName.Internal. or UserName. as prefixes for ALL packages to minimize risk, but it's incredibly important for any package with a generic name. Global uniqueness of package id-versions is a fundamental performance assumption for NuGet (and I believe also Maven, at least at the time that NuGet was first created). It's easy for me to say, since I'm not the one who will have to do the work, but I think the once-off pain of changing package IDs and updating all company internal projects to use the new package IDs is a worthwhile investment.

[raducucu82]

Also note that now Package Source Mapping is a feature that can instruct NuGet to only search specific source(s) for certain package ids or prefixes. Keep in mind, NuGet will not download a package, even when package source mapping is enabled, when the same package id-version is already in the global packages folder.

Yes, was looking into this, seems like the best approach

[space-alien]

The documentation here states that:

When multiple unique patterns match a package ID, the most specific one will be preferred. Package ID patterns always have the highest precedence while the generic * always has the lowest precedence.

It goes on further:

The * pattern can be used to make a declare a de-facto default source - meaning any package that doesn't match other specified patterns will be restored from that source without throwing an error. This configuration is advantageous if you primarily use packages from say, nuget.org, and only have a few internal packages, or use standard prefixes for all internal packages like Contoso.*.

Note that it explicitly states that the default * pattern applies when a package "doesn't match other specified patterns".

Despite all this, if a newer version of a package is found in a source that has a lower precedence match (i.e. *), that source is used, in contradiction to the documented behaviour. 🤷‍♂️

What exactly is the point of all this precedence matching if a package source with a less-specific pattern can produce a match when a newer version appears there?