NU1403: Package content hash validation failed for System.Diagnostics.Debug.4.3.0

[jozefizso]

NuGet product used: nuget.exe

NuGet version: 5.7.0 x64

VS version (if appropriate): Visual Studio 2019 16.7.3

OS version (i.e. win10 v1607 (14393.321)): Windows 10 2004 x64, Azure Hosted Agent 20200827.1

Detailed repro steps so we can see the same problem

1. Migrate projects from packages.config to PackageReference format
2. Enable RestorePackagesWithLockFile and DisableImplicitNuGetFallbackFolder in project files
3. Configure Azure Pipeline to cache nuget packages based on packages.lock.json files
4. Use nuget.exe to restore packages in Azure Pipeline build
5. Build will randomly fail with NU1403: Package content hash validation failed for System.Diagnostics.Debug.4.3.0 error message.

Other suggested things

http://thecodemanual.pl/2020/03/11/caching-not-only-nuget-packages-on-azure-devops.html
https://rafaeldossantos.net/azure-devops-pipeline-cache-nuget/

Verbose Logs

...
2020-09-14T07:56:02.8580414Z All packages and projects are compatible with .NETFramework,Version=v4.6.2 (win-x86).
2020-09-14T07:56:02.8581131Z Package content hash validation failed for System.Diagnostics.Debug.4.3.0. Expected: aMZ7dxchsUUwNeVUnDHImWs2f/5MFLbEHBktJEOJ2tYfL/a743iyuQye8qAhwVWC9hbr8O8+zfx+J4E9UTJniQ== Actual: ZUhUOdqmaG5Jk3Xdb8xi5kIyQYAA4PnTNlHx1mu9ZY3qv4ELIdKbnL/akbGaKi2RnNUWaZsAs31rvzFdewTj2g==
2020-09-14T07:56:02.8583634Z Package content hash validation failed for System.Runtime.InteropServices.RuntimeInformation.4.3.0. Expected: TbJ8Q9N8BLyOxAGTgx55HJlgsxwlWIVaY6uuqvB6/pIpR9z95qTwd1ZzNOuCqE0AcW9d8WMkjzzC3lpljK06zA== Actual: cbz4YJMqRDR7oLeMRbdYv7mYzc++17lNhScCX0goO2XpGWdvAt60CGN+FHdePUEHCe/Jy9jUlvNAiNdM+7jsOw==
2020-09-14T07:56:02.8584771Z Package content hash validation failed for System.Diagnostics.Debug.4.3.0. Expected: aMZ7dxchsUUwNeVUnDHImWs2f/5MFLbEHBktJEOJ2tYfL/a743iyuQye8qAhwVWC9hbr8O8+zfx+J4E9UTJniQ== Actual: ZUhUOdqmaG5Jk3Xdb8xi5kIyQYAA4PnTNlHx1mu9ZY3qv4ELIdKbnL/akbGaKi2RnNUWaZsAs31rvzFdewTj2g==
2020-09-14T07:56:02.8587275Z Package content hash validation failed for System.Runtime.InteropServices.RuntimeInformation.4.3.0. Expected: TbJ8Q9N8BLyOxAGTgx55HJlgsxwlWIVaY6uuqvB6/pIpR9z95qTwd1ZzNOuCqE0AcW9d8WMkjzzC3lpljK06zA== Actual: cbz4YJMqRDR7oLeMRbdYv7mYzc++17lNhScCX0goO2XpGWdvAt60CGN+FHdePUEHCe/Jy9jUlvNAiNdM+7jsOw==
2020-09-14T07:56:02.8592366Z Package content hash validation failed for System.Diagnostics.Debug.4.3.0. Expected: aMZ7dxchsUUwNeVUnDHImWs2f/5MFLbEHBktJEOJ2tYfL/a743iyuQye8qAhwVWC9hbr8O8+zfx+J4E9UTJniQ== Actual: ZUhUOdqmaG5Jk3Xdb8xi5kIyQYAA4PnTNlHx1mu9ZY3qv4ELIdKbnL/akbGaKi2RnNUWaZsAs31rvzFdewTj2g==
2020-09-14T07:56:02.8595388Z Package content hash validation failed for System.Runtime.InteropServices.RuntimeInformation.4.3.0. Expected: TbJ8Q9N8BLyOxAGTgx55HJlgsxwlWIVaY6uuqvB6/pIpR9z95qTwd1ZzNOuCqE0AcW9d8WMkjzzC3lpljK06zA== Actual: cbz4YJMqRDR7oLeMRbdYv7mYzc++17lNhScCX0goO2XpGWdvAt60CGN+FHdePUEHCe/Jy9jUlvNAiNdM+7jsOw==
2020-09-14T07:56:02.8597124Z Committing restore...
...

Full log file: nuget_restore.txt

We use this NuGet.config file:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <clear />
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
  </packageSources>
</configuration>

Sample Project

Unfortunately we don't have sample project, but this is easily reproducible when configuring Azure Pipelines with caching and nuget package restore for C# projects.

[zkat]

In theory, you should be able to try clearing the Pipelines cache. For some reason, it may have gotten polluted by someone by using an older version of NuGet that used a buggy LZMA algorithm for hashing. You might even consider disabling the Pipelines cache altogether.

Can you try one of these strategies and let me know if that works for you?

[jozefizso]

We are using the latest nuget.exe v5.7 locally and on Azure Pipeline. Cache was not polluted by older nuget release.

[jozefizso]

You might even consider disabling the Pipelines cache altogether.

It takes extremely long time to restore packages using nuget. Therefore we are caching them.

[nkolev92]

@jozefizso

Can you try clearing your global packages folder and regenerating your packages.lock.json file locally?