packages.config (PC) to PackageReference (PR) Migration

[nkolev92]

Status: In Progress

Issue to track new feature work around enabling packages.config (PC) to PackageReference (PR) upgrader workflow.

Here is the link to the spec.

Feel free to comment below with your feedback.

Related issues

#12388

Overall Issue Description:

• After enabling PackageReference support for projects, some packages may not install or work correctly #4942 After enabling PackageReference support for projects, some packages may not install or work correctly

Likely Project System Issues

• contentFiles under "any" folder doesn't work with PackageReference-style with classic csproj dotnet/project-system#3042 - ContentFiles "Any" folder issue with classic csproj

• Show contentFiles from nuget packages in legacy project system dotnet/project-system#3431 Display ContentFiles in Legacy PR

• Enable EmbedInteropTypes for PackageReference (matching Packages.Config capability) #2365 - Some assemblies in NuGet packages should be linked, not referenced (Pertains to interop assemblies and the knowledge of whether an assembly needs to be referenced. Lost when moved to the transitive world)

• Linking embedded interops in package reference  dotnet/project-system#2859 - Project System side ASK for embedded interops

• PP transforms - supported, but csproj.dll issue (link?) (see related Fix 4.3 regression - Placeholders $product$ and $AssemblyGuid$ not replaced in contentfile through transformation #5880)

• ContentFiles doesn't work with legacy PackageReference. Some issues in NuGet but also needs to coordinate with Project System as well. (ContentFiles are not added as link items in standard PackageReference #5958 PP file output is not generated during the build in classic c# project dotnet/project-system#2861)

• CPS projects can't handle contentFiles from separate packages using the same file name. (CPS projects can't handle contentFiles from separate packages using the same file name #5048)

• https://devdiv.visualstudio.com/DevDiv/NuGet/_workitems/edit/567298 - Wildcard support in legacy project system Won't fix.

• Internal bug 465204 - NuGet Build Tasks don’t put proper metadata on satellite assemblies (potentially a restore issue)

NuGet Migration Issues:

• Come up with a strategy for packages that have install.ps1/uninstall.ps1  #5963 Come up with a strategy for packages that have install.ps1/uninstall.ps1 .
• Xdt Transforms support in Package Reference #5954 XDT Transforms don't work for transitive restore
• NuGet.Config RepositoryPath controls Solution package directory for PC users. PR doesn't have a Solution package directory. (RestorePackagesPath can set the location of UserPackageFolder that is used by PC and PR codepaths) - No fix. Everything goes to the global packages folder.
• In order to pack normal PackageRef projects, you need to add a PR to NuGet.Build.Tasks.Pack, but there are still gaps (includereferencedprojects) - at that point, dotnet.exe pack will work, but nuget.exe pack won't. Related - bucket for major pack related issues #6285
• EntityFramework story - Entity Framework Pack Ref migration story #5974 & Optimize EF6 to work with PackageReference and no app.config dotnet/ef6#189
• what data should we collect to ensure we learn and adapt (this needs more thought)
• Update Package Reference Support documentation #6009 - Update documentation for package reference

NuGet PC to PR Migration Tool Issues

• Provide migration tool to move from packages.config project to packagereference #5488 - Provide migration tool to move from packages.config project to packagereference
• Add support for solution level PC -> PR migration #5879 - Add support for solution level PC -> PR migration
• Ensure the user understands changes made for them (and give them options to tweak our behavior if necessary???)
• P0 - enable project level migration (which needs to be able to walk up towards the root, changing each to PR.
  Changes to NuGet PM UI to adapt to transitive:
• Transitive dependency management, list transitive package in a flatlist - PackRef does not show transitive dependencies - install tab & update tab behavior? #5887
• Rework the UI - Improve the PC->PR Migrator UI #6530

VS Extensibility Team Issues

• VS SDK project support : NuGet packages referenced via PackageReference don't include DLLs in VSIX #5899

C++ Team Issues

• Find bug to link Internal C++ bug# 188588

WPF Project Compat Issues:

• PackageReference broken in WPF projects due to tmp_proj not importing Package-supplied build authoring #5894 - PackageReference broken in WPF projects due to tmp_proj not importing Package-supplied build authoring

Clean up our own Repro to remove the need for install.ps1

• Update NuGet.VisualStudio for PackageReference projects #5414 - Update NuGet.VisualStudio for PackageReference projects
• Understand proper design/needs for bootstrapping scenarios #5971 Enable the build bootstrapping scenario which is supported by PC today. See configure.ps1 in NuGet repo. (dotnet install tool foo.bar -- may help solve this)

Work with package owners (see related item# #5960)

• Analyze how many packages have install/uninstall.ps1 and analyze the impact this change would have.
• Analyze contentfiles vs content packages - CPS projects can't handle contentFiles from separate packages using the same file name #5048
• Analyze lib/foo.dll packages
• Consider removing the extensibility that allows other package managers to be shown as compatible #6623 Jquery/Bower + Alternative Package Manager Providers

Validation to improve ecosystem for new push/ingested packages

• once we clean up, how do we keep it clean? Validation at pack/push/ingest time?

Package Reference experience issues

• Install/Uninstall operations from PMC do not refresh PM UI for legacy project system #5939 - Install/Uninstall operations from PMC do not refresh PM UI for legacy project system
• Improve PackageReference support in old project system dotnet/project-system#2129 - Improve PackageReference support in old project system - $(MyVersion) in version string.
• Microsoft.NuGet.targets refers to project.json for PackageReference projects #4778 Error experience with unavailable frameworks, PR.
• extern alias support for NuGet package references #4989 - Documentation issue.
• PackageRef project opt-in technique #4488 - PackageRef project opt-in technique
• Allow format selection during the first package installation #8980 Allow project format selection dialog should be enabled by default.

[AArnott]

You might want to add #5894 to your list of issues to be fixed. Since NuGet contains WPF assets, this could bite you.

[nkolev92]

Done.
Thanks @AArnott.

[natidea]

As I understand it, 465204 is also a scenario that works with packages.config but not PackageReferences

[mungojam]

roslyn 22095 is one that is stopping us from migrating. I had it as a nuget issue originally. If nuget could have a type of lib that ended up being output to bin but would be flagged as not having its API exposed to consuming projects, then it wouldn't be a problem.

[AArnott]

@mungojam NuGet packages can already depend on another package without exposing the API from that package to their own consumer. I think that that problem is solved at the right place right now because, for example, if package A depends on package B, we might say the ideal situation is that package A hides package B from the compiler of the app project if and only if no APIs from package B are exposed from package A's API.

Let's take an example:

Package B defines public class Tree and Package A derives from it: public class AppleTree : Tree.
It is important then that anyone referencing Package A automatically get Package B's types so the compiler doesn't emit an error when using AppleTree telling the user to add a reference to resolve Tree as a base type. That would be a bad experience for Package A's consumer if using it was broken by default. Also, considering Package A's public API itself depends on Package B, package A cannot remove its dependency on Package B without it being a binary breaking change, so it's unlikely they would do that--at least not without a major version increment per semver.

But now consider Package A only defines internal class AppleTree (note it is internal). Package B is now an implementation detail of Package A. Now it is appropriate for Package A to specify B as a dependency in their nuspec, including the attribute to omit B as a reference to the compiler that builds the app. Now the app can't accidentally take a dependency on B without the app developer adding a PackageReference to B explicitly, thereby mitigating the problem you're calling out.

The issue that remains, I think, is that most packages out there today don't do due diligence to mark their dependencies carefully in this way. And even if a package author wanted to, it's not trivial to audit your public API for dependencies of this kind, nor to maintain that properly over time. So tooling to automatically discern between public dependencies and internal dependencies, and possibly automatically setting the appropriate package dependency metadata may be the best way to solve your problem in general.

[mungojam]

@AArnott Thanks for reviewing it, you've summed it up perfectly. I may be missing a trick then, how do we specify that a particular dependency is only to be added as an internal dependency and not exposed to compiler further up the chain?

I agree that tooling would definitely solve it nicely if the nuget mechanism exists already.

[mungojam]

@AArnott Is this the mechanism you are referring to, using contentFiles?

<contentFiles>
  <file include="bin/Release/SomeInternalDependency.dll" buildAction="None" copyToOutput="true" flatten="true" />
</contentFiles>

I might see what happens with a wildcard in the include. Maybe we could do that for the packages we make and then add external dependencies explicitly in an exclude.. sounds yukky, but might work.

[AArnott]

No, don't use contentFiles for this. When building your own package via csproj with PackageReference items, you add an attribute like this (if this were package A):

  <PackageReference Include="PackageB" Version="1.0" PrivateAssets="compile" />

This makes it so your own package A consumes and references package B in your compiler references, but folks consuming package A will not get package B in their assembly references.

Documented here

[mungojam]

Thanks, that does work, actually using "compile;contentfiles;analyzers;build" to keep the default behaviour of PrivateAssets for the other types.

So, I will raise a feature request for the tooling improvement that you suggested, recognising that it won't be an easy thing to implement.

By the way, I had tried PrivateAssets previously and hit various errors. Some my own doing (package name clash). Other issues I ran into have been raised as issues already against the project system I think. I had to keep closing VS 15.4 and deleting obj/bin/.vs folders for various changes to package references to work. I hope that is fixed soon as it is a blocker for us too.

[awesley]

Is it the case that #4491 should be included in this checklist?

[rohit21agrawal]

@awesley #6285 is tracking that work and is included in the list in the OP.

[binki]

@CoolDadTx

@binki, web.config has to be editable because devs do edit it. It is no different than an app.config in every other project type. You said making this auto-generate would solve the PR issue for ASP.NET. I fail to see how this feature (which already works) solves the problem with migrating ASP.NET apps to PR. Config files are easy to change at build time. This isn't what makes updating ASP.NET apps hard. It's the other stuff like updating bundling rules when adding content files or modifying the config file when there are user-settable options inside it (e.g. identity providers).

web.config doesn’t have to be editable. It could be built from other files, some which are user-edited and others which are transformations applied incrementally by targets from <PackageReference/>. What you’re saying is something like “web.config has to be editable because it currently is editable”. What I’m saying is that the <PackageReference/> approach would be a lot easier if Visual Studio were willing/able to run targets to generate web.config, default to web.config being in .gitignore, have a user-friendly way to store the edits that you would need to make to web.config in different files which automatically get applied to the web.config as it is generated, and public well-specified MSBuild targets that would collect information for generating web.config and run when it needs to be generated (such as after the user edits the user-editable files).

The same approaches can be extended to bundling rules. IMO, the system is broken if a file is both autogenerated and requires post-generation user edits before it works for them.

Making web.config built would make a lot of projects’ git repos a ton cleaner because they would be able to have less generated code in them. This gets us closer to the ideal of 0 boilerplate. It would allow projects’ git repos to contain exactly the code that is specific to that project and omit things which are needed for every project like FSharp assembly binding redirects (for non-ASP.NET projects, those are already autogenerated at build-time because app.config is transformed while it is being copied to bin. In ASP.NET projects, the web.config in the root of the repository has to be post-transform because that is where IISExpress looks when serving the project—which is one of the few reasons why VS still must implement the functionality where it can edit app.config/web.config for you if you right-click the error message about missing binding redirects).

[CoolDadTx]

@binki, while I agree that separating web.config into separate pieces (which you can already do by the way) would make it easier to autogenerate stuff. This doesn't really have anything to do with PR in ASP.NET projects. Your thoughts sound more like a feature request and therefore should probably be posted in the ASP.NET team. Breaking up web.config would not solve the PR issue with ASP.NET projects. It would simply lessen the things that need to be done to get it to work.

[binki]

@CoolDadTx

This doesn't really have anything to do with PR in ASP.NET projects.

I’m convinced it is a big part of it. The installation/uninstallation scripts that packages.config run primarily do things like edit app.config/web.config. Bundled assets are already solved. However, there is no way for a <PackageReference/> NuGet to edit app.config upon installation—which is a good thing. It enforces less boilerplate. .csproj files no longer generate a bunch of churn when updating or adding/removing packages to a project.

separating web.config into separate pieces (which you can already do by the way)

Where are the docs on this? I don’t expect NuGet package authors to write targets to work with one person’s way of autogenerating configuration files because there is not yet a right way to do so. Microsoft needs to make a guide on how to write a NuGet which can be installed into an ASP.NET project as <PackageReference/> and still work without requiring the user to read and manually do the steps outlined in install.ps1.

[CoolDadTx]

@binki, I'm afraid we'll have to disagree on this. Separating out the config file introduces way more problems then it solves (especially at runtime where multiple files are now needed) just to get PR to work and there are still cases where PR won't work with a package. Since ASP.NET is deprecated in lieu of ASP.NET Core (which focuses on programmatic configuration and JSON instead of web.config) I doubt MS would take the hit to implement such a feature vs just forcing everyone down the ASP.NET Core route.

Where are the docs on this?

configSource is what handles this. Some larger apps use it to separate their configurations out. Personally I find it harder to work with but I can see the draw for really large apps.

[binki]

Where are the docs on this?

configSource is what handles this. Some larger apps use it to separate their configurations out. Personally I find it harder to work with but I can see the draw for really large apps.

I know what configSource is and have looked at it. But what you linked is not a protocol for writing a <PackageReference/>-compatible NuGet package intended for consumption by ASP.NET.

Since ASP.NET is deprecated in lieu of ASP.NET Core

OK, so I am reading that is: MS does not intend to fix this issue. Thanks for the update!

[CoolDadTx]

OK, so I am reading that is: MS does not intend to fix this issue. Thanks for the update!

I don't work for MS so I cannot answer the official word on the matter but if you read all the information coming out, including .NET 5 then the writing is on the wall.

[zendu]

I see following error.

---

Microsoft Visual Studio

Operation failed

Project is not eligible for migration. Either the project is not packages.config based or doesn't support PackageReference yet. Visit https://docs.microsoft.com/en-us/nuget/reference/migrate-packages-config-to-package-reference for more information.

OK

Microsoft Visual Studio Enterprise 2022
Version 17.1.0 Preview 2.0 [31910.343.main]
VisualStudio.17.IntPreview/17.1.0-pre.2.0+31910.343.main
Microsoft .NET Framework
Version 4.8.04161

Installed Version: Enterprise

Visual C++ 2022 00476-80000-00000-AA668
Microsoft Visual C++ 2022

.NET Core Debugging with WSL 1.0
.NET Core Debugging with WSL

ASP.NET and Web Tools 2019 17.1.165.16736
ASP.NET and Web Tools 2019

ASP.NET Web Frameworks and Tools 2019 17.1.165.16736
For additional information, visit https://www.asp.net/

Azure App Service Tools v3.0.0 17.1.165.16736
Azure App Service Tools v3.0.0

Azure Functions and Web Jobs Tools 17.1.165.16736
Azure Functions and Web Jobs Tools

C# Tools 4.1.0-2.21558.9+518162cc0ccd9f90a309d4acdd5a22886865b77c
C# components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

Common Azure Tools 1.10
Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.

Fabric.DiagnosticEvents 1.0
Fabric Diagnostic Events

Microsoft Azure Service Fabric Tools for Visual Studio 17.0
Microsoft Azure Service Fabric Tools for Visual Studio

Microsoft Azure Tools for Visual Studio 2.9
Support for Azure Cloud Services projects

Microsoft JVM Debugger 1.0
Provides support for connecting the Visual Studio debugger to JDWP compatible Java Virtual Machines

Microsoft Library Manager 2.1.134+45632ee938.RR
Install client-side libraries easily to any web project

Microsoft MI-Based Debugger 1.0
Provides support for connecting Visual Studio to MI compatible debuggers

Microsoft Visual C++ Wizards 1.0
Microsoft Visual C++ Wizards

Microsoft Visual Studio Tools for Containers 1.2
Develop, run, validate your ASP.NET Core applications in the target environment. F5 your application directly into a container with debugging, or CTRL + F5 to edit & refresh your app without having to rebuild the container.

Microsoft Visual Studio VC Package 1.0
Microsoft Visual Studio VC Package

Node.js Tools 1.5.31027.1 Commit Hash:dac60d9b246a1d6a5daf23d223c933dbe1518465
Adds support for developing and debugging Node.js apps in Visual Studio

NuGet Package Manager 6.1.0
NuGet Package Manager in Visual Studio. For more information about NuGet, visit https://docs.nuget.org/

ProjectServicesPackage Extension 1.0
ProjectServicesPackage Visual Studio Extension Detailed Info

Razor (ASP.NET Core) 17.0.0.2152601+724154d925d7d9d26ebf8a73a66d5219aa320400
Provides languages services for ASP.NET Core Razor.

Snapshot Debugging Extension 1.0
Snapshot Debugging Visual Studio Extension Detailed Info

SQL Server Data Tools 17.0.62111.10130
Microsoft SQL Server Data Tools

TypeScript Tools 17.0.1029.2001
TypeScript Tools for Microsoft Visual Studio

Visual Basic Tools 4.1.0-2.21558.9+518162cc0ccd9f90a309d4acdd5a22886865b77c
Visual Basic components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

Visual F# Tools 17.1.0-beta.21558.1+467d1b5daffaf2a3f0052fc366c91b11131e6528
Microsoft Visual F# Tools

Visual Studio Code Debug Adapter Host Package 1.0
Interop layer for hosting Visual Studio Code debug adapters in Visual Studio

Visual Studio Container Tools Extensions 1.0
View, manage, and diagnose containers within Visual Studio.

Visual Studio IntelliCode 2.2
AI-assisted development for Visual Studio.

Visual Studio Tools for Containers 1.0
Visual Studio Tools for Containers

[ghost]

Issue is missing Type label, remember to add a Type label

[daniatic]

Can Microsoft please provide an official statement if PackageReference will or will not be supported by ASP.Net .NetFramework projects?

[khellang]

Under Project type support:

.NET Framework projects support PackageReference, but currently default to packages.config. To use PackageReference, migrate the dependencies from packages.config into your project file, then remove packages.config.

[caspChristian]

Under Project type support:

.NET Framework projects support PackageReference, but currently default to packages.config. To use PackageReference, migrate the dependencies from packages.config into your project file, then remove packages.config.

Only partially true, as @daniatic says ASP.Net projects currently only work with packages.config, if you try to migrate manually it fails in several ways.

See dotnet/project-system#2670

[daniatic]

I'm referring to the docs, that state:

ASP.NET apps targeting the full .NET Framework include only limited support for PackageReference. C++ and JavaScript project types are unsupported

https://learn.microsoft.com/en-us/nuget/consume-packages/package-references-in-project-files#project-type-support

Limited Support is linked to this specific issue. It is not easy to deduct, what limited support means. Is it limited support regarding the migration? Or is it limited support in general with asp.net with .net Framework? And what are these limitations? Will they be fixed? Am I supposed to read through the 91 preceding comments to get an answer? I could but I'm afraid that I won't get an thorough answer...

[khellang]

Only partially true, as @daniatic says ASP.Net projects currently only work with packages.config, if you try to migrate manually it fails in several ways.

What's the source? What are those ways?

Been using (hundreds of) PackageReference in ASP.NET WebForms apps for many years. No issues. I can't remember whether it was migrated automatically or manually (or a combination) since it's been a while, but we haven't looked back since.

See dotnet/project-system#2670

That's support for SDK-based projects. Not the same thing.

[CoolDadTx]

@khellang the two known issues that I'm aware of are config transforms and transient binding redirects. Note that you can use package references in any project but you have to be aware of the limitations as it isn't formally supported.

Config transforms used to be the biggest issue. Many packages, pre SDK, made changes to configs using Nuget's install/uninstall feature. This was dropped with package references so they can no longer make changes. The biggest package impacted by this was EF but there were many others including Dotnet Compilers. Because PRs do not run the install/uninstall scdripts then no changes are made. If the config changes were never really needed (e.g. EF) then you would never notice the issue. But Roslyn, for example, wouldn't work properly. To remedy this the package format was updated to support build files (targets and props). Most packages were updated to use this route and so you don't see the problem but older packages would continue to have problems.

Another area is transient dependencies which PRs support but package.config does not. This problem tends to rear its head when you upgrade packages. The problem comes in when you have multiple dependencies on the same third party package. You can get into versioning conflicts so you have to use binding redirects. With PRs you only need to depend on the packages your project uses and Nuget will pull in the rest but that doesn't work properly without SDK projects. In that case you can see runtime errors because the binding redirects aren't updated for transient dependencies unless you're using SDK projects. If your code happens to build in the "correct" order you may not see this problem for years but at some point an upgrade of a package or a change in build order can suddenly reveal that your project has a bad binding. The general workaround is to add the transient dependency as a direct dependency. This resolves the issue generally bad at the cost of an 'unneeded" dependency.

We too have a couple of ASP.NET projects using PRs but it is not a supported situation and we have docs that discuss how to handle the issues it causes. In at least one case we reverted back from PRs when trying to update the project at a later date because we could not get around the binding issues. So if you have it working in your projects then great for you, but it still isn't formally supported.