FileLoadException for NUnit 4.2.2 when upgrade the System.Buffers to 4.6.0

[mzh3511]

I have a test csproj based on net472, its package reference is following:

  <ItemGroup>
    <PackageReference Include="JetBrains.dotMemoryUnit" Version="3.2.20220510" />
    <PackageReference Include="MathNet.Spatial" Version="0.6.0" />
    <PackageReference Include="NSubstitute" Version="5.3.0" />
    <PackageReference Include="NUnit" Version="4.2.2" />
    <PackageReference Include="NUnit3TestAdapter" Version="4.6.0" />
  </ItemGroup>

and the unit tests runs OK

9 days ago, the System.Buffers published the 4.6.0 package, so I try it, then the test csproj's package reference changed to:

  <ItemGroup>
    <PackageReference Include="JetBrains.dotMemoryUnit" Version="3.2.20220510" />
    <PackageReference Include="MathNet.Spatial" Version="0.6.0" />
    <PackageReference Include="NSubstitute" Version="5.3.0" />
    <PackageReference Include="NUnit" Version="4.2.2" />
    <PackageReference Include="NUnit3TestAdapter" Version="4.6.0" />
    <PackageReference Include="System.Buffers" Version="4.6.0" />
  </ItemGroup>

Actully, I just append the System.Buffers 4.6.0 package, the test csproj failled many test cases, all get a FileLoadException, detail manage is following:

System.IO.FileLoadException : Could not load file or assembly 'System.Buffers, Version=4.0.3.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
   at NUnit.Framework.Constraints.Comparers.StreamsComparer.Equal(Object x, Object y, Tolerance& tolerance, ComparisonState state, NUnitEqualityComparer equalityComparer)
   at NUnit.Framework.Constraints.NUnitEqualityComparer.AreEqual(Object x, Object y, Tolerance& tolerance, ComparisonState state)
   at NUnit.Framework.Constraints.NUnitEqualityComparer.AreEqual(Object x, Object y, Tolerance& tolerance)
   at NUnit.Framework.Constraints.CollectionTally.TryRemove(Object o)
   at NUnit.Framework.Constraints.CollectionTally.<TryRemove>g__TryRemoveSlow|11_0(IEnumerable c)
   at NUnit.Framework.Constraints.CollectionEquivalentConstraint.Matches(IEnumerable actual, CollectionTallyResult& tallyResult)
   at NUnit.Framework.Constraints.CollectionEquivalentConstraint.ApplyTo[TActual](TActual actual)
   at NUnit.Framework.Assert.That[TActual](TActual actual, IResolveConstraint expression, NUnitString message, String actualExpression, String constraintExpression)
   at RanplanWireless.Terrain.Tests.RasterExtentTests.GivenBasic2x2Extent_WhenEnumerateRasterBorderCoordinates_ThenBorderReturned() in E:\workGit\iBuildNet_UK2\iBuilding\RanplanWireless.Terrain.Tests\RasterExtentTests.cs:line 139

When I downgrade the NUnit to 4.1.0 and keep the System.Buffers 4.6.0, the test csproj also runs OK

NUnitDemo.zip Here has a demo project, when remove the statement <PackageReference Include="System.Buffers" Version="4.6.0" /> from the NUnitDemo.csproj, the test will runs OK, but if the System.Buffers package reference exist, the test will be fail.

Please investigate and double-check it, thanks

[stevenaw]

Thanks for your report @mzh3511
When running in Visual Studio using the test adapter, I can confirm I can see this behaviour on net472 which your repro targets. I can't reproduce this on .NET8, indicating it could be an issue specific to when it runs on .NET Framework.

@nunit/framework-team
It can be reproduced by any test which causes the comparer chain to try the StreamsComparer, regardless of if it's needed or not. For example, since we test numerics after streams in the NUnitEqualityComparer, the following will also cause this to fail:

        [Test]
        public void Test1()
        {
            Assert.That(1, Is.EqualTo(1));
        }

Trying to apply this binding redirect:

<dependentAssembly>
	<assemblyIdentity name="System.Buffers"
	  publicKeyToken="cc7b13ffcd2ddd51"
	  culture="neutral" />
	<bindingRedirect oldVersion="0.0.0.0-4.6.0.0" newVersion="4.6.0.0" />
	<publisherPolicy apply="no" />
</dependentAssembly>

will produce the following build warning:

Found conflicts between different versions of "System.Buffers" that could not be resolved.
There was a conflict between "System.Buffers, Version=4.0.4.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51" and "System.Buffers, Version=4.6.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51".
"System.Buffers, Version=4.0.4.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51" was chosen because it was primary and "System.Buffers, Version=4.6.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51" was not.

@nunit/framework-team Assuming that binding is correct then the "chosen because it was primary" warning is curious. We don't explicitly define System.Buffers as a dependency but we do pull it in transitively via System.Memory. Should we define it?
I've pushed my testing here: https://github.com/nunit/nunit.issues/tree/main/Issue4887

[sba-schleupen]

Hello @stevenaw,
the binding redirect is working for the execution with Visual Studio but not when i execute the tests with the nunit-console.

Can we update System.Buffers or System.Memory to the new version in the nunit project?
Than everything should function again.

[stevenaw]

@sba-schleupen We're looking into it :)

@mzh3511 Does it work for you as a workaround for now to explicitly target version 4.5.5 of System.Memory instead version 4.6.0 of System.Buffers? That should at least pull in the System.Buffers reference you need in a way that at least seemed to work for me in the repro project.

For example:

  <ItemGroup>
    <PackageReference Include="JetBrains.dotMemoryUnit" Version="3.2.20220510" />
    <PackageReference Include="MathNet.Spatial" Version="0.6.0" />
    <PackageReference Include="NSubstitute" Version="5.3.0" />
    <PackageReference Include="NUnit" Version="4.2.2" />
    <PackageReference Include="NUnit3TestAdapter" Version="4.6.0" />
-   <PackageReference Include="System.Buffers" Version="4.6.0" />
+   <PackageReference Include="System.Memory" Version="4.5.5" />
  </ItemGroup>

[OsirisTerje]

@stevenaw It is the package System.Memory including System.Buffers as a dependency, which was included March 20th in PR #4668 meant to improve performance of the StreamsComparer. Version 4.1. was in February, so this was introduced in version 4.2. The version of System.Memory now is 4.5.5, and it references System.Buffers 4.5.1. But for framework .net 6, we see version 6.0.0 of the System.Buffers. So the package we refer to seems to be needed only for .net framework.

In the nuspec file we conditionally include System.Memory only for .net framework.

I can't see any good reason not to upgrade System.Memory to 4.6.0, which will also upgrade System.Buffers to the same version.

Btw: We might have a risk with System.Threading.Tasks.Extensions too, we include version 4.5.4, but the latest release is 4.6.0. Again, only .net framework.

[SFrank1966]

We encountered the same problem still with version 4.3.2 of NUnit. Our nightly build on the Devops stops some of the unit tests with the missing System.Buffers 4.0.4. Looking at the Nuget-package of NUnit 4.3.2 with dotpeek, you can see dependencies to System.Buffers 4.0.4 and System.Memory 4.0.2. I attached an image with the dotpeek output.

[manfred-brands]

Unfortunately in the System.Memory files MS has done something weird with the versioning. The NuGet version and the AssemblyVersion are out of sync.

• System.Memory

  • NuGet 4.6.0 => AssemblyVersion is 4.0.2.0
  • NuGet 4.5.5 => AssemblyVersion is 4.0.1.2

• System.Buffer

  • NuGet 4.6.0 => AssemblyVersion 4.0.4.0
  • NuGet 4.5.1 => AssemblyVersion 4.0.3.0

@SFrank1966 Does you project indirectly reference a different version of System.Buffers? I see 4.6.1 is out.

With .NET (Framework) if any referenced project uses a different version you need to explicitly redirect that one to yours.
.NET doesn't load another version of signed assemblies as it might break things.

Note that the redirect reference the AssemblyVersion, not the NuGet version!

<dependentAssembly>
	<assemblyIdentity name="System.Buffers"
	  publicKeyToken="cc7b13ffcd2ddd51"
	  culture="neutral" />
	<bindingRedirect oldVersion="0.0.0.0-4.0.5.0" newVersion="4.0.5.0" />
	<publisherPolicy apply="no" />
</dependentAssembly>

At my work we have a whole file of redirects to maintain for all shared project we use (most of them our own) as libraries update at different times. As long as we don't have major version changes with breaking changes even though annoying, it works.

@nunit/core-team We might want to reconsider removing this dependency as there will be both users that use earlier and later versions of System.Memory/System.Buffers.

[stevenaw]

Good points. I hadn't noticed a newer version of this had been released. Should we move the discussion to #4990 ? That issue is currently open and would be a good avenue to address this, potentially in the upcoming 4.4 release