WebHostBuilder.ConfigureServices method not found when using nunit3testadapter 4.4.0

[iamzhaoxu]

Describe the bug
I try to update the WireMock.Net to 1.5.17 and the nunit3testadapter to 4.4.0 in the test project and I got an exception when starting the wiremock as below

System.MissingMethodException : Method not found: 'Microsoft.AspNetCore.Hosting.IWebHostBuilder Microsoft.AspNetCore.Hosting.IWebHostBuilder.ConfigureServices(System.Action1<Microsoft.Extensions.DependencyInjection.IServiceCollection>)'. StackTrace: at WireMock.Owin.AspNetCoreSelfHost.StartAsync() at WireMock.Server.WireMockServer..ctor(WireMockServerSettings settings) at WireMock.Server.WireMockServer.Start(Nullable1 port, Boolean ssl)
Error Message:
OneTimeSetUp: System.MissingMethodException : Method not found: 'Microsoft.AspNetCore.Hosting.IWebHostBuilder Microsoft.AspNetCore.Hosting.IWebHostBuilder.ConfigureServices(System.Action1<Microsoft.Extensions.DependencyInjection.IServiceCollection>)'. Stack Trace: Failed WhenGetArticles_GivenApiReturnTwoErrorResponses_ShouldRetry [54 ms] Error Message: OneTimeSetUp: System.MissingMethodException : Method not found: 'Microsoft.AspNetCore.Hosting.IWebHostBuilder Microsoft.AspNetCore.Hosting.IWebHostBuilder.ConfigureServices(System.Action1<Microsoft.Extensions.DependencyInjection.IServiceCollection>)'.
Stack Trace:
Failed WhenGetArticles_GivenQueryMoreThanBatchSize_ShouldReturnExpected [54 ms]
Error Message:
OneTimeSetUp: System.MissingMethodException : Method not found: 'Microsoft.AspNetCore.Hosting.IWebHostBuilder Microsoft.AspNetCore.Hosting.IWebHostBuilder.ConfigureServices(System.Action`1<Microsoft.Extensions.DependencyInjection.IServiceCollection>)'.
Stack Trace: ....

After I downgrade the nunit3testadapter 4.4.0 back to 4.3.1 then the same test worked again. I am not sure this is issue from WireMock and Nunit3TestAdpter and thereby I raised this issue in both projects.

I heard that WebHostBuilder was depreciated dotnet/aspnetcore#20964 before but I still can see the .NET 7 still try to support it from https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.hosting.webhostbuilder.-ctor?view=aspnetcore-7.0.

Expected behaviour:
I will expect the latest of WireMock.Net should work with the latest nunit3testadapter

Test to reproduce
1 Refence WireMock.Net 1.5.17 and nunit3testadapter 4.4.0 in the same test project

[Merlijnv]

We are having the same issue but not using WireMock

[OsirisTerje]

@iamzhaoxu Can you add a small repro project ?
@Merlijnv What are you actually using ?

[Merlijnv]

@OsirisTerje Packages Used:
coverlet.collector 3.2.0
FluentAssertions 6.10.0
Microsoft.AspNetCore.TestHost 6.0.14
Microsoft.NET.Test.Sdk 17.5.0
NUnit 3.13.3
NUnit3TestAdapter 4.3.1
Specflow.NUnit + SpecFlow.Tools.MsBuild.Generation + SpecFlow 3.9.74

[OsirisTerje]

@Merlijnv Ok, then can you also provide a small repro for your case?

[ghelyar]

It's not just this, we are getting method not found exceptions across many of our repos on many different methods.

System.MissingMethodException : Method not found: 'System.Text.Encodings.Web.JavaScriptEncoder System.Text.Json.JsonSerializerOptions.get_Encoder()'.

System.MissingMethodException : Method not found: 'Void System.Text.Json.JsonSerializerOptions.set_Encoder(System.Text.Encodings.Web.JavaScriptEncoder)'.

System.MissingMethodException : Method not found: 'Void Microsoft.AspNetCore.Diagnostics.HealthChecks.HealthCheckOptions.set_Predicate(System.Func`2<Microsoft.Extensions.Diagnostics.HealthChecks.HealthCheckRegistration,Boolean>)'.

etc, where the only change is NUnit3TestAdapter 4.3.1 being updated to 4.4.0

This certainly affects .NET 6.0 projects, and may also affect other target frameworks.

[OsirisTerje]

@ghelyar Understand, but please give us a small repro project. Our own test projects works. I don't doubt you all have found something, but we need a repro.

[ghelyar]

I'm working to get a simple repro now.

It looks like it may only happen on Linux, as I can repro in CI and in WSL but not on Windows so far.

It also looks like it might be something to do with tests for asp.net core projects and framework references to asp.net core (e.g. project under test has <FrameworkReference Include="Microsoft.AspNetCore.App" />), but I'm still checking.

[Merlijnv]

We are also referencing a asp.net project and running Linux pipeline in Azure Devops. Maybe @OsirisTerje could reproduce it that way?

[OsirisTerje]

I have added a repro project using Wiremock.net (sample from wiremock.net), see https://github.com/nunit/nunit3-vs-adapter.issues/tree/master/Issue1065/wiremocktests

This repro works, on my Windows machine.
It does not include the <FrameworkReference Include="Microsoft.AspNetCore.App" /> , and I have not tested it on Linux. So, the smallest repro that you see fail would be appreciated. Please also confirm it works on Windows, but on Linux (or none of these ofc.)

I am also using 4.4.0 on a work project with asp.net core and have no issues there.

Please also be aware that the adapter have no connections to asp.net core at all, so if it is affected by that, it has to be some sideffects of something.

[martin-shields-sage]

I can confirm we have the same issues here as well, our error message is different though:

OneTimeSetUp: System.BadImageFormatException : Could not load file or assembly 'System.Transactions.Local, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. Reference assemblies should not be loaded for execution.  They can only be loaded in the Reflection-only loader context. (0x80131058)
  ----> System.BadImageFormatException : Could not load file or assembly 'System.Transactions.Local, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51, processorArchitecture=MSIL'. Reference assemblies should not be loaded for execution.  They can only be loaded in the Reflection-only loader context. (0x80131058)
  ----> System.BadImageFormatException : Cannot load a reference assembly for execution.

[OsirisTerje]

Checked the wiremock repro I have - on WSL, still works, so it is not Linux alone.

[OsirisTerje]

@martin-shields-sage Repro ? And, can you check this using the NUnit.Console ? Also, Windows or Linux?

[martin-shields-sage]

I am using WSL (Ubuntu), just debugging through now to see why it is happening exactly

[ghelyar]

Here's a simple repro of one of the cases I have seen:
https://gist.github.com/ghelyar/2c85b4d264a586602fdd357c8723b193

replace underscores in the file names with slashes for the directory structure.

run dotnet test tests/tests.csproj from Linux (e.g. WSL) with .NET 6+ installed (tested with SDKs 6.0.406 and 7.0.201)

works with 4.3.1, fails with 4.4.0 on Linux

works with both 4.3.1 and 4.4.0 on Windows

It's also possible that rather than OS it's actually which SDKs are installed etc, but I have not been able to narrow that down yet.

[OsirisTerje]

@ghelyar Thanks! What does dotnet --list-sdks on WSL show ?
Sorry, saw your list in the gists now.....

[OsirisTerje]

@ghelyar I do get a failure, but not the same one:

And it works on Windows, but not on WSL, so thanks! Then I have something to work on.

[nesc58]

Same error here.

Multitarget project fails with NUnit3TestAdapater 4.4.0 fails with .net6.0 on linux and works fine on windows.
.net7.0 works fine.

Example error with 4.4.0 on linux with .net6.0
System.MissingMethodException : Method not found: 'Void Microsoft.Extensions.Configuration.ConfigurationBinder.Bind(Microsoft.Extensions.Configuration.IConfiguration, System.Object)'

System.MissingMethodException : Method not found: 'Microsoft.AspNetCore.Hosting.IWebHostBuilder Microsoft.AspNetCore.Hosting.IWebHostBuilder.ConfigureServices(System.Action`1<Microsoft.Extensions.DependencyInjection.IServiceCollection>)'

Using NUnit3TestAdapter 4.3.1 everything works like a charm (.net7.0.201, net6.0.14 on linux and windows)

[ghelyar]

@ghelyar I do get a failure, but not the same one:

And it works on Windows, but not on WSL, so thanks! Then I have something to work on.

This is actually the same error I was getting, it's just the 3rd error in my original message:

System.MissingMethodException : Method not found: 'Void Microsoft.AspNetCore.Diagnostics.HealthChecks.HealthCheckOptions.set_Predicate(System.Func`2<Microsoft.Extensions.Diagnostics.HealthChecks.HealthCheckRegistration,Boolean>)'.

and then wrapped in an assert throws because I used an exception to cut the repro short.

---

here's another, even simpler repro:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.5.0" />
    <PackageReference Include="NUnit" Version="3.13.3" />
    <PackageReference Include="NUnit3TestAdapter" Version="4.4.0" />
    <PackageReference Include="System.Text.Json" Version="7.0.2" />
  </ItemGroup>

</Project>

using NUnit.Framework;

namespace tests;

[TestFixture]
public class Class1
{
    [Test]
    public void Test()
    {
        var options = new System.Text.Json.JsonSerializerOptions()
        {
            Encoder = System.Text.Encodings.Web.JavaScriptEncoder.UnsafeRelaxedJsonEscaping
        };
    }
}

This one fails on both Windows and Linux for me on NUnit3TestAdapter 4.4.0, but passes on NUnit3TestAdapter 4.3.1 for both.

System.MissingMethodException : Method not found: 'Void System.Text.Json.JsonSerializerOptions.set_Encoder(System.Text.Encodings.Web.JavaScriptEncoder)'.

However, it only fails if I include System.Text.Json 7.x as a package reference on a .NET 6 project. If I just let it use the framework's built in System.Text.Json, or use the package version 6.0.7, it works fine on NUnit3TestAdapter 4.4.0.

The Encoder property was added in .NET Core 3.0, so it should be present in both.

[martin-shields-sage]

apologies for jumping in again, debugging through my issue I get stuck at the point where the exception is thrown, looking at the inner exception it looks like the adapter attempts to load in assemblies

at System.Runtime.Loader.AssemblyLoadContext.LoadFromPath(IntPtr ptrNativeAssemblyLoadContext, String ilPath, String niPath, ObjectHandleOnStack retAssembly)\n   at System.Runtime.Loader.AssemblyLoadContext.LoadFromAssemblyPath(String assemblyPath)\n   at NUnit.Engine.Internal.TestAssemblyResolver.OnResolving(AssemblyLoadContext context, AssemblyName name)\n   at NUnit.Engine.Internal.TestAssemblyLoadContext.Load(AssemblyName name)\n   at System.Runtime.Loader.AssemblyLoadContext.ResolveUsingLoad(AssemblyName assemblyName)\n   at System.Runtime.Loader.AssemblyLoadContext.Resolve(IntPtr gchManagedAssemblyLoadContext, AssemblyName assemblyName)

the assembly it is looking for is System.Transactions.Local which is weird because I don't see any reference existing anywhere for this. Nor does it exist in the grand scheme of things (we have not created it and Microsoft dont have an assembly of Transactions which has .Local on the end from what I can find)

[iamzhaoxu]

Hi, @OsirisTerje I noticed

1. When you add Microsoft.Extensions.Hosting 7.0.1, WireMock.Net 1.5.17 and nunit3testadapter 4.4.0 as a dependency in the project and you keep the project TargetFramework as .net6.0, if you run the test on Linux OS, the error can be reproduced. You can give it a try with your sample https://github.com/nunit/nunit3-vs-adapter.issues/tree/master/Issue1065/wiremocktests . The same scenario can run successfully on Windows OS.

2. If you change the TargetFramework to .net7.0, both Linux and Windows will run successfully.

3. When you downgrade the nunit3testadapter to 4.3.1 and you keep the project TargetFramework as .net6.0,, both Linux and Windows will run successfully.

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
    <Nullable>enable</Nullable>

    <IsPackable>false</IsPackable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.Extensions.Hosting" Version="7.0.1" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.11.0" />
    <PackageReference Include="NUnit" Version="3.13.2" />
    <PackageReference Include="NUnit3TestAdapter" Version="4.4.0" />
    <PackageReference Include="coverlet.collector" Version="3.1.0" />
    <PackageReference Include="WireMock.Net" Version="1.5.17" />
  </ItemGroup>

</Project>

[OsirisTerje]

@iamzhaoxu @ghelyar @nesc58 @martin-shields-sage @Merlijnv
Please find attached an alpha version of 4.4.1-alpha.1 and a nuget config file. The alpha version works now on @paulhickman-a365's repro from issue #1066. Can you all please verify that it also works on your solutions?
This alpha uses the 3.15.2 engine, but extended as a 3.15.3-dev engine with added support for .net 8. That is a "Point in time" where I am pretty sure we have a stable version. It will thus not have the fixes that are in the 3.16.X engine, but looking through the issues there I think it will work. The 4.3.1 adapter is based on 3.15.2 engine.
I suggest you add the package to a c:\nuget folder locally, and then add the nuget.config (in the nuget.zip file) to your solution, so that the package is found there.
NUnit3TestAdapter.4.4.1-alpha.1.zip
nuget.zip

[dickwolff]

@OsirisTerje Can confirm the fix works locally.

[nesc58]

For me it works with the 4.4.1-alpha.1.

I have another question:
What is the Microsoft.Extensions.DependencyModel dependency? It is new since 4.4.0. Is this really required?

[martin-shields-sage]

@OsirisTerje can confirm here this works locally for me as well

[ghelyar]

It also works locally for me

[OsirisTerje]

@dickwolff @nesc58 @martin-shields-sage @ghelyar Thanks for confirming!

@nesc

What is the Microsoft.Extensions.DependencyModel dependency? It is new since 4.4.0. Is this really required?

This comes from the NUnit.Engine 3.16.2 and is used to load deps.json files. It looks like this is the reason for the reflection load issues. In the new alpha it is removed, since the alpha is based on a dev version 3.15.3 of the engine. So, short answer, it is probably the reason for the crashes, and yes, it is removed.

That said, in order to get out a 4.4.1 we need to release an engine 3.15.3 first. I'll get that process started first, and hope to have this done real soon.
What can be done is to release the alpha prerelease on nuget.org, but it might then not be compatible with any NUnit.Console, except perhaps with the 3.1.5.2 console. The NUnit.Console is not that much used, so it may be an acceptable compromise.

[OsirisTerje]

Hotfix version 4.4.2, nearly identical to the alpha version attached yesterday, is now released. The difference from the attached version is that the Microsoft.Extensions.DependencyModel is not include in the adapter package. It is not needed there, so it was removed. The release notes is here https://docs.nunit.org/articles/vs-test-adapter/AdapterV4-Release-Notes.html (includes some more explanations) and the package is uploaded to nuget https://www.nuget.org/packages/NUnit3TestAdapter/4.4.2. Thanks, everyone for reporting and providing repros!