This repository has been archived by the owner. It is now read-only.

Test Discovery Error with ASPNET Core 1.1 #99

Open
Robula opened this Issue Jan 16, 2017 · 7 comments

Comments

Projects
None yet
3 participants
@Robula

Robula commented Jan 16, 2017

There seems to be an issue that causes test-nunit to fail to discover tests when "Microsoft.AspNetCore.Mvc": "1.1.0" is added to the test project either directly or via another project reference.

Interestingly, removing the dependency to "Microsoft.AspNetCore.Mvc": "1.1.0" or simply lowering the version ("Microsoft.AspNetCore.Mvc": "1.0.2") allows my tests to be discovered.

Here is my project.json:

{
  "version": "1.0.0-*",

  "dependencies": {
    "NUnit": "3.6.0",
    "NUnit3TestAdapter": "3.6.0",
    "dotnet-test-nunit": "3.4.0-beta-3",
    "Moq": "4.6.38-alpha",
    "Microsoft.AspNetCore.Mvc": "1.1.0"
  },

  "testRunner": "nunit",

  "frameworks": {
    "netcoreapp1.1": {
      "dependencies": {
        "Microsoft.NETCore.App": {
          "version": "1.1.0",
          "type": "platform"
        }
      }
    }
  }
}

The error I get when I build and "Run All" tests is this:

------ Discover test started ------
NUnit Adapter 3.6.0.0: Test discovery starting
Error: Unable to get runner for this assembly. Check installation, including any extensions.
FileNotFoundException: Could not load file or assembly 'nunit.framework' or one of its dependencies. The system cannot find the file specified.
Dependent Assembly nunit.framework of D:\project\bin\Debug\netcoreapp1.1\Project.Tests.dll not found. Can be ignored if not a NUnit project.
NUnit Adapter 3.6.0.0: Test discovery complete
========== Discover test finished: 0 found (0:00:00.1140183) ==========
------ Discover test started ------
Discovering tests in 'D:\project\Project.Tests\project.json' ["C:\Program Files\dotnet\dotnet.exe" test "D:\project\Project.Tests\project.json" --output "D:\project\Project.Tests\bin\Debug\netcoreapp1.1" --port 15591 --parentProcessId 206184 --no-build]
'test-nunit' returned '-2'.
========== Discover test finished: 0 found (0:00:00.823869) ==========

'test-nunit' returned '-2'. resolves to INVALID_ASSEMBLY.

@rprouse

This comment has been minimized.

Show comment
Hide comment
@rprouse

rprouse Jan 16, 2017

Member

FYI, this likely won't be fixed because Microsoft is dropping support for the dotnet-test-* style adapters, so this project will be deprecated soon.

As a workaround, does the dotnet test command work?

Member

rprouse commented Jan 16, 2017

FYI, this likely won't be fixed because Microsoft is dropping support for the dotnet-test-* style adapters, so this project will be deprecated soon.

As a workaround, does the dotnet test command work?

@Robula

This comment has been minimized.

Show comment
Hide comment
@Robula

Robula Jan 17, 2017

Hi @rprouse, thanks for your help.
I didn't realise they were dropping support for the adapters, does this mean that NUnit will no longer support/work with .NET Core?

dotnet test does appear to discover the tests but fails elsewhere

NUnit .NET Core Runner 3.4.0
Copyright (C) 2016 Charlie Poole
Could not load file or assembly 'Microsoft.DotNet.InternalAbstractions, Version=1.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified.
SUMMARY: Total: 1 targets, Passed: 0, Failed: 1.

Robula commented Jan 17, 2017

Hi @rprouse, thanks for your help.
I didn't realise they were dropping support for the adapters, does this mean that NUnit will no longer support/work with .NET Core?

dotnet test does appear to discover the tests but fails elsewhere

NUnit .NET Core Runner 3.4.0
Copyright (C) 2016 Charlie Poole
Could not load file or assembly 'Microsoft.DotNet.InternalAbstractions, Version=1.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified.
SUMMARY: Total: 1 targets, Passed: 0, Failed: 1.

@rprouse

This comment has been minimized.

Show comment
Hide comment
@rprouse

rprouse Jan 17, 2017

Member

@Robula NUnit will support .NET Core again, it will just take us a bit of time to play catch-up and add in support in our other adapter 😄

As for your error, I need to look at it more closely, but my guess is that you are directly or indirectly using a different version of Microsoft.DotNet.InternalAbstractions than the adapter is. Unfortunately, .NET Core does not have support for AppDomains, so we cannot isolate your tests from the adapter code in a separate AppDomain like we do in desktop.

Member

rprouse commented Jan 17, 2017

@Robula NUnit will support .NET Core again, it will just take us a bit of time to play catch-up and add in support in our other adapter 😄

As for your error, I need to look at it more closely, but my guess is that you are directly or indirectly using a different version of Microsoft.DotNet.InternalAbstractions than the adapter is. Unfortunately, .NET Core does not have support for AppDomains, so we cannot isolate your tests from the adapter code in a separate AppDomain like we do in desktop.

@Robula

This comment has been minimized.

Show comment
Hide comment
@Robula

Robula Jan 18, 2017

Thanks @rprouse, you've been most helpful! I'll look forward to the full .NET Core support, in the meantime I'll fall back to MSTest for any new .NET Core projects.

Robula commented Jan 18, 2017

Thanks @rprouse, you've been most helpful! I'll look forward to the full .NET Core support, in the meantime I'll fall back to MSTest for any new .NET Core projects.

@rprouse

This comment has been minimized.

Show comment
Hide comment
@rprouse

rprouse Jan 18, 2017

Member

It stumbled across a blog post where xUnit is having the same problem with Microsoft.DotNet.InternalAbstractions. MSTest probably has the same issue. I haven't tested the fix yet, but I would appreciate feedback if it works.

The blog post is http://ardalis.com/xunit-test-discovery-error-with-aspnet-core-1-1

The fix is to explicitly add Microsoft.DotNet.InternalAbstractions version 1.0.0 as a dependency to your project.json

Member

rprouse commented Jan 18, 2017

It stumbled across a blog post where xUnit is having the same problem with Microsoft.DotNet.InternalAbstractions. MSTest probably has the same issue. I haven't tested the fix yet, but I would appreciate feedback if it works.

The blog post is http://ardalis.com/xunit-test-discovery-error-with-aspnet-core-1-1

The fix is to explicitly add Microsoft.DotNet.InternalAbstractions version 1.0.0 as a dependency to your project.json

@rprouse rprouse changed the title from test-nunit fails to discover tests when a particular dependency is added to Test Discovery Error with ASPNET Core 1.1 Jan 18, 2017

@MrMartinRoy

This comment has been minimized.

Show comment
Hide comment
@MrMartinRoy

MrMartinRoy Jan 27, 2017

We had the same problem here. And yes, MSTest has the same issue.

Above fix worked for us.

MrMartinRoy commented Jan 27, 2017

We had the same problem here. And yes, MSTest has the same issue.

Above fix worked for us.

@rprouse

This comment has been minimized.

Show comment
Hide comment
@rprouse

rprouse Jan 27, 2017

Member

Thanks for confirming the fix.

Member

rprouse commented Jan 27, 2017

Thanks for confirming the fix.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.