dotnet test with .NET 8 SDK installed

[csim]

• NUnit 3.13.1, NUnit3TestAdapter 4.3.0
• Visual Studio 2022 enterprise edition 17.8.1

After installing .NET 8, running "dotnet test" throws the error:

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
NUnit Adapter 4.3.0.0: Test execution started
Running selected tests in C:\_work\1\s\bin\published-tests\Release\NetDesktop\net45\Microsoft.ProgramSynthesis.Transformation.Table.Tests.dll
Exception System.TypeInitializationException,    Exception thrown executing tests in C:\_work\1\s\bin\published-tests\Release\NetDesktop\net45\Microsoft.ProgramSynthesis.Transformation.Table.Tests.dll
The type initializer for 'NUnit.Engine.Services.RuntimeFrameworkService' threw an exception.
   at NUnit.Engine.Services.RuntimeFrameworkService.ApplyImageData(TestPackage package)
   at NUnit.Engine.Services.RuntimeFrameworkService.SelectRuntimeFramework(TestPackage package)
   at NUnit.Engine.Runners.MasterTestRunner.GetEngineRunner()
   at NUnit.Engine.Runners.MasterTestRunner.Explore(TestFilter filter)
   at NUnit.VisualStudio.TestAdapter.NUnitEngine.NUnitEngineAdapter.Explore(TestFilter filter) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnitEngine\NUnitEngineAdapter.cs:line 88
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunAssembly(String assemblyPath, IGrouping`2 testCases, TestFilter filter) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 275
InnerException: System.ArgumentException: Unknown framework version 8.0
Parameter name: version
   at NUnit.Engine.RuntimeFramework.GetClrVersionForFramework(Version frameworkVersion)
   at NUnit.Engine.RuntimeFramework..ctor(RuntimeType runtime, Version version, String profile)
   at NUnit.Engine.RuntimeFramework.GetNetCoreRuntimesFromDirectoryNames(IEnumerable`1 dirNames)
   at NUnit.Engine.RuntimeFramework.FindDotNetCoreFrameworks()
   at NUnit.Engine.RuntimeFramework.get_AvailableFrameworks()
   at NUnit.Engine.Services.RuntimeFrameworkService..cctor()
NUnit Adapter 4.3.0.0: Test execution complete
No test matches the given testcase filter `TestCategory = LearningPerformance & TestCategory != ExcludeWindows & TestCategory != LLM & TestCategory != Experiment` in C:\_work\1\s\bin\published-tests\Release\NetDesktop\net45\Microsoft.ProgramSynthesis.Transformation.Table.Tests.dll

Upgrading NUnit3TestAdaptor is not possible because the test project targets net45 which is not supported by the NUnit3TestAdapter package starting in version 4.3.1.

Unless I've missed something, it appears as though it is not possible to use "dotnet test" for a test project targeting net45 on a machine that has .NET 8 installed with NUnit3TestAdapter 4.3.0

I can understand deprecating old framework versions, but could we get a 4.3 patch to enable "dotnet test" for machines that have .NET 8 installed?

[stevenaw]

Thanks for the report @csim
To help us understand, is there a reason the tests must target net45 as a compilation target?

[OsirisTerje]

@csim You are correct, that is compatibility at its best [not] ;-)

As @stevenaw says, you should be able to run the test project at a higher framework version than your project under test. That could solve this.

But, we could also do as you suggest and release a 4.3.2 version which uses the 3.15.4 engine. That would solve the issue wrt net8. That should be a very minor release change. We will see what we can do wrt to that.

I assume you're also aware the the test sdk has to be below version 17.4, because that also has a cutoff at framework 4.6.2.

Please also note that the support for net45 also apply for 4.3.1. The cutoff is at 4.4

UPDATE:
We need to get out a fix for this. I tried running it under a lower dotnet sdk, using a gobal.json file to set to net 6, but it still crashes. This version of the engine actually enumerates ALL versions, and ignores the global.json file. That is not good.
I also tried to override the 3.13.2 with a package reference directly to 3.15.4, but it doesn't override it, and that is surely due to the way we have the engine embedded here.

[OsirisTerje]

@csim I built a 4.3.2-alpha.1 package, enclosed here in the zip file. Copy it down to a local folder (e.g. c:\nuget), add/edit a nuget.config to have a key pointing to that folder, and update the csproj. Tried on a local repro and it worked here.

Please confirm.
If it works for you, I'll take it through bureaucracy and get it released during the weekend.

Example nuget.config:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <clear />
    <add key="Local" value="C:\nuget" />
    <add key="nuget" value="https://api.nuget.org/v3/index.json"/>
  </packageSources>
</configuration>

NUnit3TestAdapter.4.3.2-alpha.1.zip

[OsirisTerje]

Released 4.3.2, on nuget now

[csim]

Thank you so much!

We produce an SDK and our consuming partners are locked on old framework versions. We want to ideally test against old framework versions to guarantee compatibility. This change will enable us to move forward with things .NET 8 and still test against old versions.

We are working to integrate the new adaptor package version 4.3.2

[csim]

Hi Folks, it looks like I may have miscommunicated the exact ask here.

The old framework deprecation happened in the adapter transition from v4.3.0 -> v4.3.1. I mistakenly assumed that the v4.3 chain was compatible with net45 and netcoreapp2.1.

Because of this we can't use the v4.3.1 or v4.3.2 version of the adaptor on our legacy testing projects. Perhaps enabling .NET 8 in this version will help others.

We actually need a patch to v4.2.1 to enable .NET 8. Would that be possible?

Thanks again for all your help on this issue and sorry for the rehash.

[OsirisTerje]

4.3.2 support down to net framework 3.5, that is it includes 4.5.
With netcore, it does not support netcore 2.1, lowest is netcore 3.1

Also have a look at #1026 . This popped up and was the cause for the 4.3.1.

It would then be the netcore 2.1 that is your challenge. And, you can target anything, from netcore 3.1, so that means all you have to do is to set the test project to netcore 3.1, and then keep your classlibs at netcore 2.1. That will still work.

I am skeptical of moving down to 4.2. The further back we go, the higher risk of a major fault. The 4.3 adapter series uses the 3.15 engine series. The 4.2 uses the 3.13 engine series. I am not sure if we can upgrade that.
Also, I am personally running out of time now, I have 3 more days and then I leave for a month.

1. Can you check if you can upgrade the testproject alone to netcore 3.1?
2. Alternative (if 1 fails) : You can clone the adapter repo, and do the exact same changes I did for the the upgrade to 4.3.2. Then you can test that locally. If that works you can raise a PR. I will bring along a laptop, so I can approve a PR, and possibly manage to do a release too.

This is commit with the changes I did for 4.3.2, as you can see they are very small: b3263eb

The thing is to run all the tests, and ensure this is actually working.

PS: About the issue above, I noted in my repro here that I started to get InternalTrace log files again.

[csim]

Thanks for all the information, we will try to avoid netcorapp2.1 for now.

[OsirisTerje]

@csim May I ask, what kind of software is this ? Libraries or application?

Btw, here is the link to the repro project I made based on your information above.
https://github.com/nunit/nunit3-vs-adapter.issues/tree/master/Issue1144

PS: If this is critical for you, I can manage to get out a 4.2.2, with no warranty that it will be working in all aspects though, but it may be end of next week.

UPDATE:
Sorry, I didnt see your earlier post here, saying it is SDKs.

I'll see what I can do the next few days, but I am travelling from Sun - Wedn,, so it may be end of next week.