It should be easier to effectively use unreleased builds of the .NET SDK

[baronfel]

Is your feature request related to a problem? Please describe.

Today folks that want to use nightly builds of the SDK (or even use the redist build here from the repo, etc) face two fundamental problems:

• getting the bits
• getting the bits configured with the correct feeds so that our long-tail of NuGet-package-delivered functionality lights up
  • this is things like workloads, self-contained publishing, trimming, AOT, single-file hosts, apphosts, etc

Today, the 'get the bits' problem is at least somewhat mitigated by

• the dotnet/dotnet VMR builds table
• the dotnet-install scripts nightly feeds
• the internal release-tracker tool (for private builds, security release previews, and so on)

In the future, some people (it's me, I'm the people) would like SDK PRs to generate bundles that would be usable for testing/validation purposes - but that's just an expansion of the VMR use case for the purposes of this discussion.

Regardless of how users get the bits, they need to do additional NuGet feed configuration in order for all of our implicit PackageReferences and PackageDownloads to work. This often hits Visual Studio dogfooders when we insert an unreleased SDK into VS ahead of public release for overall alignment purposes.

Describe the solution you'd like

The actual goal is the SDK bits you download should be able to access a pre-configured feed, and that feed should be configured to only server/impact the use of the .NET SDK's implicit PackageReferences/PackageDownloads

One implementation towards this goal could be:

• expanding the compositional lookup hierarchy of the NuGet.config file today from "machine, then user, then solution/repo" to "machine, then toolchain, then user, then solution/repo"
  • essentially allowing the .NET SDK layout/install a place to influence source configuration/resolution
  • this would need design and agreement from the @dotnet/nuget-team, naturally, though it may provide a more 'official' or 'natural' way for concepts like the Visual Studio fallback feed to be configured.
  • 'toolchain' for the dotnet CLI environment would be the SDK install root, and for the VS environment would be some other location that VS installations can control. there should be a design/agreement around this.
• making the SDK build process (in the SDK repo, and then eventually in the VMR) emit a NuGet.config file that specifies
  • the correct package source(s) that serve every implicitly-used package the SDK toolchain may insert into a project's dependency hierarchy
  • package source mapping entries to ensure those implicitly-used packages are only served from those preconfigured feeds
• depending on the usage of the SDK in question (SDK repo PR build, VMR nightly, final stable release, VS insertion), the generate NuGet.config may or may not be included in the actual SDK layout:
  • SDK repo PR build: included
  • VMR nightly build: included
  • final stable release before public release: not included, but generated and easily gettable? maybe by tooling?
  • final stable release (or servicing previews): not included
  • VS insertion (of stable releases or servicing releases): not included in the SDK, but could potentially be included as a component?

Contents of the generated NuGet.config

An SDK manipulates the following kinds of packages that should be sourced from the non-NuGet.org feeds in this scenario

• workload packs
• workload manifests
• runtime packs
• targeting packs
• actual NuGet packages

Many of these have a well-known set of name patterns or are knowable statically, so we should be able to relatively easily know which feed(s) they should be sourced from.

Let's consider a single pack - the linux-x64 AppHosts pack Microsoft.NETCore.App.Host.linux-x64 across net8/9/10 TFMs across the scenarios above:

scenario	TFM	feed
SDK PR	NET10	dotnet-10
SDK PR	NET9	dotnet-9
SDK PR	NET8	dotnet-8
VMR nightly	NET10	dotnet-10
VMR nightly	NET9	dotnet-9
VMR nightly	NET8	dotnet-8
Stable release prior to public	NET10	release-specific dotnet-10 prep feed
Stable release prior to public	NET9	release-specific dotnet-9 prep feed
Stable release prior to public	NET8	release-specific dotnet-8 prep feed
Stable release	NET10	N/A - defer entirely to user config
Stable release	NET9	N/A - defer entirely to user config
Stable release	NET8	N/A - defer entirely to user config
VS Insertion of unreleased	NET10	release-specific dotnet-10 prep feed
VS Insertion of unreleased	NET9	release-specific dotnet-9 prep feed
VS Insertion of unreleased	NET8	release-specific dotnet-8 prep feed

This is highly pattern-based and would be fairly easy to template out and generate during the build.

The generated NuGet.config would look something like:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <packageSources>
        <clear />
        <add key="dotnet-10" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet10/nuget/v3/index.json" />
        <add key="dotnet-9" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet9/nuget/v3/index.json" />
        ...
    </packageSources>

  <packageSourceMapping>
      <packageSource key="dotnet-10">
        <package pattern="Microsoft.NETCore.App.Host.linux-x64" />
      </packageSource>
      <packageSource key="dotnet-9">
        <package pattern="Microsoft.NETCore.App.Host.linux-x64" />
      </packageSource>
      ...
    </packageSourceMapping>
</configuration>

By explicitly mapping the packages the SDK will automatically provide, we limit the scope of usage of these feeds in whatever end-user configurations that may exist.

The thinking here is that for the most painful use cases today we can actually know what the correct feeds are - and potentially with some release tracker work we can also make the NuGet.config for the stable-but-not-yet-released SDKs more easily usable by internal partners.

Open Questions

Template packages

Problem: @DamianEdwards brings up templates that we ship and how those often include PackageReferences that would also need to be updated.

Solution: Add mapping entries for Microsoft.* and System.* to the generated config as well. For PRs/Nightlies these would probably point to dotnet-libraries and/or dotnet-tools, and for internals we'd point to the per-release dedicated feeds. This makes it more important that we use explicit packageId mapping entries for the runtime packs, workloads, etc so that we don't reintroduce dependency confusion.

Global package directory dirtying

Problem: These nuget.configs would introduce package versions that aren't available on end-user package sources to the global package cache. That could cause confusion if users switch between preview/nightly SDKs and stable SDKs for their 'real' work.

Solution: generated nuget.configs should use a separate global packages cache to prevent this dirtying.

Drawbacks

• NuGet.config location would need expansion.
• NuGet.config allows removal of package sources (and maybe PSM data? need clarification on that), so it's possible and maybe even likely that the generated config would still conflict with user repo configurations. That's ok though - users that customize their repo NuGet configuration should be able to navigate this, and we can doc the behavior/experience.
• It would force us to reconcile with the set of packages we work with implicitly
  • this is actually a benefit, though - we should know this and track it

[richlander]

I like this direction. This comes up all the time, particularly near the end of a release. For folks who deal with this every day, it is second nature, but for everyone else, it requires a decoder ring.

My only concern with this is if we can make it work reliably. I grant that it would work super well for nightly builds. There is this period where we produce release candidates (not RCs but "candidates for release" which could be preview, RC, GA, or servicing) that may end up being the final build. We don't have the opportunity to have an extra file tht points to alternative feeds in that case. I don't have an easy intuition on how to solve that. Perhaps there could be a CLI command that could generate a local nuget.config from a URL that you have access to. However, that's similar to what we already have. It's also possible that I've lost the plot on the actual problem to solve.

[baronfel]

I'm definitely hand waving a bit on that second scenario! My thought process is something like:

• enable making an artifact (the config file) that solves the problem
• start generating the artifact
• embed the artifact when we can (PR/Nightly)
• for the more gnarly scenarios the problem is now 'how do I get ARTIFACT_X' instead of 'nothing works, help!'

[DamianEdwards]

Love the idea! Making these SDK builds easier to consume will reduce a lot of friction for our own dogfooding (something we found on the Aspire team when making similar investments).

Another package-kind to consider is packages referenced by templates included in the SDK, e.g. the ASP.NET Core templates often include package references that will use versions aligned with the channel/quality the SDK was built from. So, if I get a daily SDK build, then run dotnet new webapi, the created project will reference the Microsoft.AspNetCore.OpenApi package, using a version that is only available on the relevant feed. Without handling this, projects created from these SDKs using dotnet new may not be usable without custom nuget configuration. In the Aspire CLI case, we resolved this by having the aspire new command create an appropriately configured nuget.config to include in the new project artifacts but given the proposal here is to make the toolset feed an SDK intrinsic, that might not be needed.

Speaking of templates, one of the templates in the SDK is one for a nuget.config file. It would be neat if the nugetconfig template in a given SDK build included the relevant feed for that SDK build (along with nuget.org).

[richlander]

This makes me think that a channel concept would be useful. You could then acquire nuget.config and other files from that channel outside the SDK. I'm not keen on mutating the actual product to include this info.

[baronfel]

@DamianEdwards I'm assuming that the Microsoft.* and System.* packages our templates depend on would be in dotnet-libraries or dotnet-tools, right? If we were explicit about the runtime packs, targeting packs, workload manifests, etc. then I think it would be fairly easy (at least for SDK/VMR builds) to add a package source mapping for Microsoft.* and System.* to that feed.

For the 'internal' builds those packages would be in the private, per-release feed like all the other artifacts, so the mapping would go to that feed instead.

I'm not certain about how to handle the nugetconfig template. At minimum we'd want to update the SDK template snapshot-based tests to not error in the presence of this new information - but I also wonder for nightlies/PRs/etc if it'd be easier to distribute the templates in nuget-package form instead of baked-into-the-sdk form.

One other thing I slightly worry about is polluting user global packages directories with this stuff - I wonder if a channel-based/PR/etc-based nuget.config should set a different global packages path to prevent impacting 'real' data stores.

Updated the description with both of these open questions.

[DamianEdwards]

@baronfel some of them come from the regular dotnetX (where X is the version) feeds I believe.

Channel-based restore location is nice, but IIRC there might be some complications with folks who use the NUGET_PACKAGES environment variable to move their global package location to somewhere like a Dev Drive, as the environment variable takes precedence over configuration (from NuGet docs):

[baronfel]

That is rough - I suppose for those folks they would fall into one of the drawbacks I'd called out already (emphasis mine):

NuGet.config allows removal of package sources (and maybe PSM data? need clarification on that), so it's possible and maybe even likely that the generated config would still conflict with user repo configurations. That's ok though - users that customize their repo NuGet configuration should be able to navigate this, and we can doc the behavior/experience.

So for their case, wouldn't the more stable/preferred solution (slash mitigation for your identified problem) be to set the global packages path in a user-level NuGet.config? (Which yes would still have the precedence problem given that my proposed layering is

• machine
• toolchain
• user
• repo

sigh
)

[DamianEdwards]

Honestly the only time this is really an issue is when using a candidate stable build (one with no pre-release suffix) before it's officially released. This happens quite rarely (shortly before release) so I'm not sure it's worth trying to ensure the overall design caters for it the same way as consuming daily builds, etc. If we do decide to configure unique package restore locations for a given SDK build, we could consider printing a warning on relevant actions if the environment variable is detected to make it clear to the user what's happening.

[marcpopMSFT]

We've done passes on ideas like this in the past (albeit without package source mapping to limit the exposure) and the challenge is that for most releases, the feed you need is either unique or locked down (or both). When we're in RC1/RC2, the downlevel feed is a stable feed. When in servicing, it's a stable feed. Theoretically we know at build time what the feed might be but the feed isn't actually created as part of the build (it's created as part of staging so doesn't even exist at build time).

@blowdart has also had a lot of concerns in the past in making more of these feeds available more easily because you have to take into account what if the release is an MSRC. In other words, what would you have done for RC2/October servicing? I also share Rich's concerns of modifying the product for this scenario.

Other options we've considered in the past is making it easier for the customer to overload the expected version, making the expected version come from elsewhere (runtime install, a runtime pack, etc). They all have drawbacks. We have not found a solution yet that seems to work well enough to pursue.

[DamianEdwards]

Theoretically we know at build time what the feed might be but the feed isn't actually created as part of the build (it's created as part of staging so doesn't even exist at build time).

We do this in the Aspire CLI, i.e. pre-calc what the feed will be and burn it into the channel feed config so a nuget.config can be dropped.

@blowdart has also had a lot of concerns in the past in making more of these feeds available more easily because you have to take into account what if the release is an MSRC. In other words, what would you have done for RC2/October servicing? I also share Rich's concerns of modifying the product for this scenario.

Obscurity isn't a protection, and these feeds are internal and require authn/z anyway, plus SDKs built from those pipelines aren't publicly available anyway.