Question: Nunit3 with Mono under Ubuntu

[jcurl]

My Open Source targets frameworks .NET 4.0-.NET 4.8, .NET Standard 2.1. On Windows I use the VSTestAdapter 17.4.0, and recently worked around the fact that .NET 4.6.2 is the lowest version (I build for .NET 4.0, but link with .NET 4.8 and have hacks for that).

When running Under Linux, with Mono, on Ubuntu 22.04, everything builds, but I would like to know what are the best practices for running .NET 4.8 under Mono/Linux. I get errors such as:

Testhost process for source(s) '/home/jcurl/source/rjcp.base/framework/nunitextensions/CodeQualityTest/bin/Debug/net462/RJCP.CodeQualityTest.dll' exited with error: Cannot open assembly '/usr/lib/dotnet/sdk/7.0.108/Extensions/../TestHostNetFramework/testhost.exe': No such file or directory.
. Please check the diagnostic logs for more information.

or

Testhost process for source(s) '/home/jcurl/source/rjcp.base/framework/crashreporter/CrashReporterTest/bin/Debug/net48/RJCP.Diagnostics.CrashReporterTest.dll' exited with error: Cannot open assembly '/usr/lib/dotnet/sdk/7.0.108/Extensions/../TestHostNetFramework/testhost.net48.exe': No such file or directory.
. Please check the diagnostic logs for more information.

Any best practices to resolve this?

My test project package references have:

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.6.2" />
    <PackageReference Include="NUnit" Version="3.13.3" />
    <PackageReference Include="NUnit.Analyzers" Version="3.6.1" />
    <PackageReference Include="NUnit.ConsoleRunner" Version="3.16.3" />
    <PackageReference Include="NUnit3TestAdapter" Version="4.5.0" />
    <ProjectReference Include="..\Xml\RJCP.Core.Xml.csproj" />
    <ProjectReference Include="..\..\nunitextensions\CodeQuality\RJCP.CodeQuality.csproj" />
  </ItemGroup>

To execute, I run:

dotnet test --nologo -c Debug --no-build --logger=\"console;verbosity=quiet\" --logger=\"trx\" RJCP.Framework-Linux.sln

I could ignore the .NET framework tests all together with -f netcoreapp3.1 (note I have to set the env var DOTNET_ROLL_FORWARD=LatestMajor`. but that skips over Mono .NET Framework tests. And this isn't scalable as soon as I start mixing test targets like netcoreapp3.1 and net6.0 (I'm trying to have a common environment for Linux and Windows, development on Windows, test on Linux).

I include the console runner, I can run that and execute the tests from within Mono on the target. Does that mean dotnet test can't execute that? Is there a suggestion of a target, or changes to my .csproj files to execute nunit3-console instead of using the VS Test Runner (it's not that great, but needed for testing in VS2019/VS2022).

Thank you for your time and a great test framework. My FOSS is at github.com/jcurl/RJCP.base (and all the submodules).

[OsirisTerje]

Hi! I'm struggling to understand the issue here.
Are you actually running the standard adapter or have you modified it? And did you also modify the other components, engine and framework?
Or, doesn't the standard components work for you using net 4.6.2 ?

You also mention including the console runner - that can be scary business, depending on how you install it and what version you include.

Can you provide a small repro for this?

[jcurl]

Thanks for taking the time to read the issue. I've attached a small example.
test.zip

I have Mono Mono JIT compiler version 6.12.0.182, .NET Core 7.0.108 [/usr/lib/dotnet/sdk] and its runtime Microsoft.NETCore.App 7.0.8 installed on Ubuntu 22.04.

Running the test program gives the error:

$ export DOTNET_ROLL_FORWARD=LatestMajor
$ dotnet test
  Determining projects to restore...
  All projects are up-to-date for restore.
  nunitproj -> /home/jcurl/source/test/test/bin/Debug/net48/RJCP.MyLibraryTest.dll
  nunitproj -> /home/jcurl/source/test/test/bin/Debug/netcoreapp3.1/RJCP.MyLibraryTest.dll
  nunitproj -> /home/jcurl/source/test/test/bin/Debug/net7.0/RJCP.MyLibraryTest.dll
Test run for /home/jcurl/source/test/test/bin/Debug/net48/RJCP.MyLibraryTest.dll (.NETFramework,Version=v4.8)
Microsoft (R) Test Execution Command Line Tool Version 17.4.1+79d56b02b69a582cd90428878a5e9411ab7538f5 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Testhost process for source(s) '/home/jcurl/source/test/test/bin/Debug/net48/RJCP.MyLibraryTest.dll' exited with error: Cannot open assembly '/usr/lib/dotnet/sdk/7.0.108/Extensions/../TestHostNetFramework/testhost.net48.exe': No such file or directory.
. Please check the diagnostic logs for more information.

Test Run Aborted.
Test run for /home/jcurl/source/test/test/bin/Debug/netcoreapp3.1/RJCP.MyLibraryTest.dll (.NETCoreApp,Version=v3.1)
Microsoft (R) Test Execution Command Line Tool Version 17.4.1+79d56b02b69a582cd90428878a5e9411ab7538f5 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.

Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1, Duration: 11 ms - RJCP.MyLibraryTest.dll (netcoreapp3.1)
Test run for /home/jcurl/source/test/test/bin/Debug/net7.0/RJCP.MyLibraryTest.dll (.NETCoreApp,Version=v7.0)
Microsoft (R) Test Execution Command Line Tool Version 17.4.1+79d56b02b69a582cd90428878a5e9411ab7538f5 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.

Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1, Duration: 11 ms - RJCP.MyLibraryTest.dll (net7.0)

See

Testhost process for source(s) '/home/jcurl/source/test/test/bin/Debug/net48/RJCP.MyLibraryTest.dll' exited with error: Cannot open assembly '/usr/lib/dotnet/sdk/7.0.108/Extensions/../TestHostNetFramework/testhost.net48.exe': No such file or directory.
. Please check the diagnostic logs for more information.

My question is, what is the best practice to resolve this error on Linux, that my nunit tests can be tested on Linux using the Mono runtime?

[OsirisTerje]

I am no mono/linux expert, but:
Are you sure you have the correct .net framework installed?
Since the .net core test is working, which it should, the error looks to be in missing the .net framework.
See if this post can help: https://andrewlock.net/building-net-framework-asp-net-core-apps-on-linux-using-mono-and-the-net-cli/ and also this one dotnet/sdk#335

[jcurl]

Yes, I am sure the correct frameworks are installed. The binaries generated targeting .NET 4.x work fine with mono. So can the .Net Core when prefixed with dotnet. I can even run the tests under Linux by hosting them with NUnit Console runner.

installation of Mono is from the Mono Project, not the default packages provided by Ubuntu 22.04 that they’ve traditionally been buggy.

it is only using the VSTest target that dotnet Test uses which doesn’t know how to run NUnit on Linux, because it appears the test host is a native windows binary, and not a .Net binary, hence the observed failures.

i appreciate the links (the GitHub issue appears irrelevant but is quite long) so I’ll investigate them further and experiment.

[jcurl]

I had a further look, and the links posted don't help. They're related to targetting frameworks and builds. There is nothing wrong with the build and it works fine. The only point is that the sample project posted here in test.zip builds, but does not know how run nunit test cases on Linux with Mono installed.

As an alternative, I'm looking now into creating my own targets to execute nunit3-console.exe which will run under Mono. Running on the command line does execute the tests. This has its own problems, but at least now I've got some prototypes working (nunit3 isn't good at detecting runtimes automatically, so this is where most of my work will go into, mapping the target framework to the --framework option, e.g. when building for .NET Core 3.1, but only having .NET 7.0 installed).

So if anyone knows how to get the Microsoft Test SDK working on modern Ubuntu 22.04 with Mono installed (64-bit only, 6.12.199), that would be helpful.

[jcurl]

Testing last night, with .NET SDK 8.0 from MS on Ubuntu 22.04, it runs the test cases using Mono. So it appears that this was a broken feature in 17.x when .NET 4.0 was removed, and appears to be fixed. Not sure if it's worthwhile to follow up on this any more.