Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Packages in FallbackFolders installed by .NET Core SDK are custom installed, and fail signature validation. #7414

Closed
livarcocc opened this issue Oct 19, 2018 · 36 comments
Labels
Functionality:Signing Resolution:External This issue appears to be External to nuget Type:Bug
Milestone

Comments

@livarcocc
Copy link

From @AArnott on September 10, 2018 23:18

As discussed on this appveyor forum, AppVeyor builds recently started breaking for me. Feodor Fitsner kindly came up with a minimal repro that suggests that the .NET Core 2.1 packages on nuget.org have bad signatures.

Repro

Create a test-nuget.csproj file with these contents:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFrameworks>netcoreapp1.0;netcoreapp2.0;netcoreapp2.1</TargetFrameworks>
  </PropertyGroup>
</Project>

Rename or remove your %userprofile%\.nuget\packages folder.

Restore packages for test-nuget.csproj using msbuild:

msbuild /t:restore /p:RestoreDisableParallel=true,TreatWarningsAsErrors=true

This produces these errors:

  Restoring packages for D:\temp\test-nuget\test-nuget.csproj...
<snip/>
  Installing Microsoft.NETCore.DotNetAppHost 2.0.0.
  Installing Microsoft.NETCore.App 2.1.0.
D:\temp\test-nuget\test-nuget.csproj : error NU3005: The package signature file entry is invalid. The central directory header field 'compression method' has an in
valid value (8).
  Installing Microsoft.NETCore.DotNetHostPolicy 2.1.0.
D:\temp\test-nuget\test-nuget.csproj : error NU3005: The package signature file entry is invalid. The central directory header field 'compression method' has an in
valid value (8).
  Installing Microsoft.NETCore.Platforms 2.1.0.
D:\temp\test-nuget\test-nuget.csproj : error NU3005: The package signature file entry is invalid. The central directory header field 'compression method' has an in
valid value (8).
  Installing Microsoft.NETCore.Targets 2.1.0.
D:\temp\test-nuget\test-nuget.csproj : error NU3005: The package signature file entry is invalid. The central directory header field 'compression method' has an in
valid value (8).
  Installing NETStandard.Library 2.0.3.
  Installing Microsoft.NETCore.DotNetHostResolver 2.1.0.
D:\temp\test-nuget\test-nuget.csproj : error NU3005: The package signature file entry is invalid. The central directory header field 'compression method' has an in
valid value (8).
  Installing Microsoft.NETCore.DotNetAppHost 2.1.0.
D:\temp\test-nuget\test-nuget.csproj : error NU3005: The package signature file entry is invalid. The central directory header field 'compression method' has an in
valid value (8).
  Generating MSBuild file D:\temp\test-nuget\obj\test-nuget.csproj.nuget.g.props.
  Generating MSBuild file D:\temp\test-nuget\obj\test-nuget.csproj.nuget.g.targets.
  Restore failed in 32.27 sec for D:\temp\test-nuget\test-nuget.csproj.

Interestingly, this only repros while there are multiple target frameworks specified. Reducing it to just netcoreapp2.1 eliminates the error.

Copied from original issue: dotnet/sdk#2523

@livarcocc
Copy link
Author

@rohit21agrawal @nkolev92 any idea what would cause these errors?

@livarcocc
Copy link
Author

Also adding @mmitche @leecow and @vitek-karas in case there is something wrong with the packages we pushed to nuget.org.

@livarcocc
Copy link
Author

From @rohit21agrawal on September 10, 2018 23:21

CC: @dtivel @PatoBeltran for signing issues

@livarcocc
Copy link
Author

From @nkolev92 on September 11, 2018 1:18

Did some digging cause this looks weird, if you do a NuGet.exe verify -signatures on the package, for example Microsoft.NETCore.App 2.1.0, you get the same error message.

This seems to suggest that there is something wrong with this package.

The reason why it fails with multitargetting is because of the fallback folders.
When you cross-target 2.x and 1.x, the new fallback folder from C:\Program Files\dotnet\sdk\NuGetFallbackFolder is not used.

When you target 2.x only, then the package is consumed from the fallback folder, instead of downloading it and installing in the global packages folder.

tl;dr;

  • The multi-targeting scenario is by design because NuGet needs to download that package.
  • Rest of it suggests that the package is not signed correctly.

@dtivel @PatoBeltran will be able to give more info on the 2nd one.

@livarcocc
Copy link
Author

From @mmitche on September 11, 2018 15:35

This is really odd. Why would it start now? 2.1.0 is ages old. Was there a change in nuget?

@livarcocc
Copy link
Author

From @mmitche on September 11, 2018 15:50

WIth nuget.exe 4.6.2.5055 I get a successful verification

@livarcocc
Copy link
Author

From @PatoBeltran on September 12, 2018 0:23

Did some investigation in this with @nkolev92. It seems like the fallback folder in C:\Program Files\dotnet\sdk\NuGetFallbackFolder is being used as a source and the package in there has the signature entry compressed.

This repros with any version of nuget greater than 4.6.2 (4.6.2 has a worse error experience, but the package errors out). If you test this with the same package from nuget.org the verification succeeds, therefore this tells us that the packages in the fallback folder are the ones that are bad.

NuGet tools do not create any compressed signatures, which tells us that there was something that modified the packages after they where signed.

@livarcocc is sdk doing anything with the packages after signing that could have modified the compression metadata of the signature entry in the packages?

@livarcocc
Copy link
Author

The fallback folder is used as a source if your application targets netcoreapp 1.1 or 1.0. Yes, the lzma compresses all files, however, we have been doing that for quite some time now.

@PatoBeltran we are going to need help to figure out what we are doing wrong here. Is it a file that we are not expanding correctly?

@livarcocc
Copy link
Author

From @jainaashish on September 12, 2018 18:41

We also ran an additional exercise to validate packages hash from extracted SHA512 file vs generating SHA512 hash from nupkg in dotnet fallback folder and all these package validation failed which suggest either dotnet is not extracting nupkg correctly in fallback folder or they are creating a new nupkg in fallback folder.

The right expectation here is that dotnet has single nupkg for each of their package, which is extracted into fallback folder using NuGet and keep the same nupkg there or publish the same nupkg to NuGet.org or any other feed. At no point in time, there should be another nupkg generated for the same package (on the fly or anything) after the original nupkg has been extracted in fallback folder.

@livarcocc
Copy link
Author

From @AArnott on October 19, 2018 0:19

What's happening here? Now I'm seeing this error in VS itself, blocking intellisense and builds:

Error occurred while restoring NuGet packages: The package signature file entry is invalid. The central directory header field 'compression method' has an invalid value (8).

@livarcocc
Copy link
Author

I am going to move this issue over to NuGet, where we can get quicker and better answers to this.

@mmitche
Copy link

mmitche commented Oct 19, 2018

@livarcocc Let me know if you need anything from me.

@dtivel
Copy link
Contributor

dtivel commented Oct 19, 2018

Somewhere something modified the package after it was signed. It's now considered a tampered package.

@AArnott
Copy link
Contributor

AArnott commented Oct 22, 2018

From the nuget side, can the error message be improved to actually mention:

  1. what package ID and version failed?
  2. where the package was being read from?

Then as a user I might be able to speculate as to how to workaround the problem.

@dtivel
Copy link
Contributor

dtivel commented Oct 22, 2018

I think this has been done already for NuGet 4.9 / VS 15.9. See NuGet/NuGet.Client#2360

@AArnott
Copy link
Contributor

AArnott commented Oct 22, 2018

@livarcocc I'm not sure why this issue was moved to github considering github isn't the team that deployed the faulty packages with the VS or dotnet SDK installer. Who "owns" this issue at this point?

@jainaashish
Copy link
Contributor

@AArnott there are two parts of the issue with the same root cause.

First part is when you started seeing this as AppVeyor builds which was failing to verify some packages signature which was a legit issue since dotnet lzma tool recompress and generates a new nupkg which failed the signature verification.

Second part is when you started seeing this inside VS as well, this is happening because NuGet introduced a new nupkg metadata file called .nupkg.metadata to contain the package's content hash to differentiate between existing SHA512 file which contains .nupkg file hash. So when NuGet tries to generate this file for an already extracted package in global packages folder, it fails to get the content hash for the signed package (this is the same failure as in the first case).

For first part, we've already improved the error message to show the corresponding package id, version, source to provide more details about the failure. And also, once lzma tool refresh the packages from NuGet.org, then these failures will be fixed (possibly in 15.9 preview5).

Second part is bit tricky and needs to resolve the upgrade scenario of moving from invalid signed package to signed package. We're still discussing and exploring multiple ways to improve this and get the right experience. But at least will make sure to add package id, version details with this exception as well. For now, one workaround is to clean global packages folder and run restore.

@AArnott
Copy link
Contributor

AArnott commented Oct 22, 2018

For now, one workaround is to clean global packages folder and run restore.

Thanks, @jainaashish. By "global packages folder" are you referring to %userprofile%\.nuget\packages?

@nkolev92 nkolev92 added this to the 4.9 milestone Oct 22, 2018
@jainaashish
Copy link
Contributor

Yes. If you aren't overwriting it in your project.

@rrelyea rrelyea modified the milestones: 4.9, 4.9.x Nov 20, 2018
@rrelyea rrelyea changed the title Builds broken because package signature file entry is invalid Packages in FallbackFolders installed by .NET Core SDK are custom installed, and fail signature validation. Nov 27, 2018
@rrelyea rrelyea added the Resolution:External This issue appears to be External to nuget label Nov 28, 2018
@tillig
Copy link

tillig commented Nov 29, 2018

Just updated VS to 15.9.3 which came with .NET SDK 2.1.500 and started getting this error. Even clearing %userprofile%\.nuget\packages didn't solve it for me; I had to rename the C:\Program Files\dotnet\sdk\NuGetFallbackFolder to something else and create a new, empty C:\Program Files\dotnet\sdk\NuGetFallbackFolder. After that and running restore... finally I could build again.

@nkolev92
Copy link
Member

@tillig
You would need to somehow get a valid version of the faulty package into the global packages folder before trying to restore again.

Simply deleting the global packages folder, and restoring the same project again would get you in the same situation.

@PatoBeltran
Copy link

This issue was because the LZMA tool used by the dotnet sdk didn't reconstruct the packages correctly and made the scenarios where the sdk fallback folders are used as a source (any netcore 1.x) to fail signature validation. Note that this is not an issue in netcore 2.x since the fallback folder is actually used as a fallback folder.

Since dotnet is not officially supporting 1.x and also 3.x will eliminate the use of the LZMA tool there is no much work left to do here. Will close the issue. Feel free to reopen if there is similar behavior in the future.

@mikkelbu
Copy link

mikkelbu commented Feb 23, 2019

Hi @PatoBeltran. In the NUnit project we have run into this problem on AppVeyor (see error in the end of this comment), see nunit/nunit-console#556 for more context.

My guess is that it is due to an update of nuget to version 4.9.2 (from version 4.7.1, if https://www.appveyor.com/updates/ is correct). We have prior solved the same issue on AppVeyor by introducing a separate nuget restore step (instead of calling the MSBuild restore target), see nunit/nunit-console#525. But now this new step is also failing.

We still, however, support netcore 1.x, so I've been trying to disable/remove the fallbackfolder when building on AppVeyor, but so far I've not succeeded. I've tried

  • Specifying the following in our NuGet.config
  <fallbackPackageFolders>
    <clear />
  </fallbackPackageFolders>
  • Specifying the following in our csproj
<RestoreFallbackFolders>clear</RestoreFallbackFolders>

Do you know any way we can disable/remove the fallbackfolder (perhaps as separate nuget operation before the nuget restore)?

Example failing build

...
Errors in C:\projects\nunit-console\src\NUnitEngine\mock-assembly\mock-assembly.csproj
    NU3005: Package 'Microsoft.NETCore.Platforms 2.0.0' from source 'C:\Program Files\dotnet\sdk\NuGetFallbackFolder': The package signature file entry is invalid. The central directory header field 'compression method' has an invalid value (8).
...

@mikkelbu
Copy link

Ps. We solved the problem by inserting the following element in the project file

<DisableImplicitNuGetFallbackFolder>true</DisableImplicitNuGetFallbackFolder>

@ctaggart
Copy link

This continues to be problematic for those of us trying to use NuGet lock files with dotnet 2.x. I think the best solution is to:

  1. delete the NuGetFallbackFolder
  2. disable recreation of the NuGetFallbackFolder
  3. clear out all NuGet caches
  4. clear our your project
  5. recreate lock files

On Windows, I did this with:

  1. deleted C:\Program Files\dotnet\sdk\NuGetFallbackFolder
  2. [Environment]::SetEnvironmentVariable("DOTNET_SKIP_FIRST_TIME_EXPERIENCE", "1", "User")
  3. dotnet nuget locals all --clear
  4. ⚠️ git clean -xfd (commit or back up anything needed but not committed)
  5. dotnet restore --force-evaluate

For Linux, it may be something like:

  1. rm -rf /usr/share/dotnet/sdk/NuGetFallbackFolder/ (use dotnet --info to find the sdk)
  2. put export DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1 in ~/.bash_profile
  3. (same)
  4. (same)
  5. (same)

Nate has details at https://natemcmaster.com/blog/2018/02/02/dotnet-global-tool/ for DOTNET_SKIP_FIRST_TIME_EXPERIENCE gotchas.

@forki
Copy link

forki commented Jan 2, 2020

I still see this error with .NET SDK 3.1.100

@rrelyea
Copy link
Contributor

rrelyea commented Jan 2, 2020

@forki - would love details of that.
Does that machine have 3.1.100 and 2.x.xxx both installed?
Does the repro machine use any global.json?
Does the problem only occur with TargetFrameworks that are supported by 2.x?

@forki
Copy link

forki commented Jan 2, 2020 via email

@forki
Copy link

forki commented Jan 2, 2020

Looks like I'm not the only one. https://twitter.com/TBHolschen/status/1212816843863404546?s=19

But the error message is really from hell. It should tell me where the issue is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Functionality:Signing Resolution:External This issue appears to be External to nuget Type:Bug
Projects
None yet
Development

No branches or pull requests