Stack overflow with paket update on new project, apparently while resolving version conflicts from Nuget.Packaging 5.7

[rmunn]

Description

Running dotnet paket update on a brand-new project using FAKE produced a StackOverflowException while trying to resolve dependencies. I created the project using the MiniScaffold template (dotnet new -i MiniScaffold then dotnet new mini-scaffold -n NewProj -ou console), then ran dotnet tool update paket and dotnet tool restore to ensure I was running the latest version of Paket (5.249.2 as of this writing). I then edited the paket.dependencies file and removed the specific version numbers that the MiniScaffold template added, so that I could let Paket find the latest version of each package. But running dotnet paket update ended up overflowing the stack while trying to resolve conflicts. I ran Paket in verbose mode to produce a log file, and I'm attaching a gzipped copy of the log file to this issue. The full log file is 22 MB, which gzipped down to 308 KB. (Using .zstd got it down to 104 KB for an amazing 99.5% compression ratio, but GitHub issues don't allow .zstd attachments so I had to use gzip.)

Repro steps

Please provide the steps required to reproduce the problem

1. Install .NET SDK 3.1.402

2. Clone https://github.com/rmunn/repro-paket-stackoverflow

3. Run dotnet tool restore

4. Run dotnet paket update and note the stack overflow

It's possible that the stack overflow is due to a combination of Nuget package versions and requirements that exists now but will go away in the future, so hopefully the attached log file will help even if the repro stops working at some point.

Expected behavior

Paket completes without overflowing the stack.

Actual behavior

Stack overflow after about a minute of working. (See attached paket-stackoverflow.log.gz file).

Known workarounds

The stack overflow appears to have something to do with trying to resolve the many available preview versions of System.Security.Cryptography.Cng 5.0.0-preview.3 and up (all the way up to preview.8 as of this writing). These appear to be required by Nuget.Packaging 5.7, which is an indirect dependency of Fake.DotNet.Paket (which was in my paket.dependencies file). Adding nuget Nuget.Packaging < 5.7 to paket.dependencies appears to resolve the stack overflow.

[baronfel]

You will also pull in this bad package version of Nuget.Packaging by adding a dependency to Fake.Dotnet.Cli, which has a transitive dependency on Nuget.Packaging through the Fake.Dotnet.Nuget package. Pinning the version to < 5.7 as you've specified fixes the problem here as well, for the same underlying reasons.

[CumpsD]

https://www.nuget.org/packages/NuGet.Packaging/5.7.0 is the latest stable, which sadly enough references 2 preview packages:

System.Security.Cryptography.Cng (>= 5.0.0-preview.3.20214.6)
System.Security.Cryptography.Pkcs (>= 5.0.0-preview.3.20214.6)

It seems Paket probably doesn't want to pull in those preview packages, causing the resolution to fail

The fix above allows install and update to work, but this will still fail: dotnet paket outdated --ignore-constraints;

This would fix the last command: dotnet paket outdated --ignore-constraints --include-prereleases;

... update: nevermind, it doesn't seem to work, the last command doesn't crash anymore, but it also doesn't ignore contraints anymore with the include-prereleases option

[Dolfik1]

@forki this issue should be fixed by #4059