Visual Studio 2017 RC does not detect nunit tests #305

CharliePoole · 2017-02-23T21:59:30Z

@claunia commented on Thu Feb 23 2017

Hi,

Visual Studio 2017 RC does not detect nunit tests. Package references NUnit 3.6.0, VS has NUnit 3 Test Adapter 3.7.0.0 installed, and project targets both .NET 4.0 and .NET Core RC4.

Running manually nunit from console or GUI works (targetting the .NET 4.0), but gets no debug backtrace (showing simply NullReferenceException on the test but not the backtrace of where it really happened).

@JustinRChou commented on Thu Feb 23 2017

If I am not wrong, the test runner for 2017 isn't ready yet.

#297

@CharliePoole commented on Thu Feb 23 2017

The adapter supports VS 2017 but not .NET Core as yet. See the issue referenced by @JustinRChou

@claunia commented on Thu Feb 23 2017

@CharliePoole read again, my project targets BOTH .NET Framework 4.0 not .NET Core. The tests are not detected even if I remove the multitargetting and leave .NET Framework 4.0 alone.

@JustinRChou commented on Thu Feb 23 2017

@claunia
It's an issue with Visual Studio 2017 itself.

The test explorer in the IDE is having issues since the NUnit test adapter hsan't been updated for that IDE.

@CharliePoole commented on Thu Feb 23 2017

@JustinRChou The NUnit 3 Test Adapter is developed and tested using VS2017. In fact, it's not possible to build it with VS2015 or lower any longer.

@claunia Am I correct to assume we are talking about the NUnit 3 Visual Studio Adapter and not the NUnit dotnet runner? I'd like to move the problem to the correct repository.

@claunia commented on Thu Feb 23 2017

@CharliePoole I'm pretty much old-fashioned, I'm not using dotnet but VS only.

This is exactly what I did:
1.- Remove old csproj
2.- Create new csproj specifying .NET Core Library on VS2017
3.- Changed netcoreapp1.0 to net40;netcoreapp1.0
4.- Added NuGet reference to NUnit 3.6.0 inside VS2017 menu
5.- Added NUnit Adapter 3.7.0 in VS2017 Extensions menu
6.- Tried changed net40;netcoreapp1.0 to net40 as suggested in #297 that .NET Core was not supported

@CharliePoole commented on Thu Feb 23 2017

@claunia It's possible that starting out with a .NET core project has led to the problem. Things you may try:

Verify that the adapter is being run at all. If it is, there will be messages in the Test output window.
Start over with a project that only targets 4.0.

I'm moving this to the adapter repository.

claunia · 2017-02-23T22:32:39Z

@CharliePoole I get no output from the tests output window.

CharliePoole · 2017-02-23T23:55:35Z

That would indicate either that it's not installed or that VS isn't calling it for some reason.

ChrisMaddock · 2017-02-24T11:41:14Z

I was under the impression that the adapter doesn't currently work with the new csproj format? There's a workaround on the issue below however:

#300

CharliePoole · 2017-02-24T13:42:35Z

Right... but I was under the impression that it doesn't always use that format, depending on how you create the project. That's what I meant in suggesting that @claunia recreate it. Am I wrong? Do all new VS2017 projects use the new format?

NZSmartie · 2017-02-26T12:14:07Z

I'm hitting the same issues here with a project that was originally a .net core Project.
The "Tests" output claims that nunit,framework assembly wasn't found

------ Discover test started ------
NUnit Adapter 3.7.0.0: Test discovery starting
Dependent Assembly nunit.framework of C:\Users\nzsma\Documents\Visual Studio 2017\Projects\CoAP.Net\CoAP.Net.Tests\bin\Debug\netstandard1.6\CoAP.Net.Tests.dll not found. Can be ignored if not a NUnit project.
Assembly contains no NUnit 3.0 tests: C:\Users\nzsma\Documents\Visual Studio 2017\Projects\CoAP.Net\CoAP.Net\bin\Debug\netstandard1.3\CoAP.Net.dll
NUnit Adapter 3.7.0.0: Test discovery complete
========== Discover test finished: 0 found (0:00:00.108052) ==========

I downloaded the NUnit.Framework-3.6.0.zip and extracted the contents to the output directory and got a slightly different error:

------ Discover test started ------
NUnit Adapter 3.7.0.0: Test discovery starting
Dependent Assembly System.Reflection, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a of C:\Users\nzsma\Documents\Visual Studio 2017\Projects\CoAP.Net\CoAP.Net.Tests\bin\Debug\netstandard1.6\CoAP.Net.Tests.dll not found. Can be ignored if not a NUnit project.
Assembly contains no NUnit 3.0 tests: C:\Users\nzsma\Documents\Visual Studio 2017\Projects\CoAP.Net\CoAP.Net\bin\Debug\netstandard1.3\CoAP.Net.dll
NUnit Adapter 3.7.0.0: Test discovery complete
========== Discover test finished: 0 found (0:00:00.1190869) ==========

This to me suggests that required assemblies needed to run the tests aren't loading or aren't being loaded from the correct place.

NZSmartie · 2017-02-26T12:15:35Z

The project is configured at a .Net Standard 1.6 Library. So I doubt Visual Studio 2017 has any reason to include required assemblies into the output build folder

qqqwwweeea · 2017-03-04T11:37:38Z

Any update on the issue? When can we expect it to be fixed?
We have a project using nunit and now it's a showstopper for us

AlexeiScherbakov · 2017-03-08T17:35:06Z

I have similar issue in release, but workaround is working

Visual Studio Config: VS 2017 + NUnit Test Adapter 3.7.0
project with multitargets netstandard1.4 & net40
test project (class library) that targets net40
At first run I got System.OutOfMemoryException, on others runs I got nothing

I target test project to net45 and add this:
<PropertyGroup> <DebugType Condition="'$(TargetFramework)' != '' AND '$(TargetFramework)' != 'netcoreapp1.0'">Full</DebugType> </PropertyGroup>

JessePreiner · 2017-03-29T15:10:41Z

I'm sure no one is as daft as I am, but I was having this issue and the reason was that my test project had a reference to Nunit2 (I was so sure that upgraded to 3 on all projects that it took me a long while to check this), and therefore the Nunit3 adapter wasn't picking up any tests. I removed the reference and installed nunit3, that did it for me.

Edit: Visual Studio Professional 2017 15.0.0+26228.10 | .NET 4.6

jhickson · 2017-03-31T05:23:54Z

@ManyouRisms The issue is with .NET Core test projects and not those purely targeting .NET 4.x.

qrli · 2017-04-04T10:36:35Z

Thanks to @AlexeiScherbakov 's great workaround. Some explanation (my understanding though) for those who encounter this:

The new .NET Core .csproj by default uses the new portable PDB as . However, many 3rd party tools do not support this new format yet. So they try to parse it as the old PDB but crashes. E.g. I also got error when using DotCover to compute coverage. So the current workaround is to set it to the same as old .csproj, which is Full for debug build and PdbOnly for release build.

However, VS Code only supports portable PDB.

grokky1 · 2017-05-24T13:21:09Z

@AlexeiScherbakov @qrli Must I use the DebugType workaround in the test project (FooTests.dll), or the project under test (Foo.dll), or both?

jnm2 · 2017-05-24T16:58:56Z

@grokky1 Only the test assembly needs <DebugType>Full</DebugType>.

rprouse · 2017-05-24T16:59:19Z

@grokky1 AFAIK, you need the debug type workaround if you are using the new CSPROJ format with the Full .NET Framework. If you are targeting .NET Core or .NET Standard, you need to use the 3.8.0-alpha1 release of the adapter, see https://github.com/nunit/nunit3-vs-adapter/releases/tag/3.8-alpha1

If you are using full .NET Framework with the original CSPROJ format, you shouldn't need to do anything special.

You will need to give us more info to help you pinpoint your exact issues.

grokky1 · 2017-05-24T17:14:00Z

@rprouse My projects are .NET Core 1.1.2, and I'm using VS2017, csproj, and the new adapter.

Tests are discovered only when I use the workaround.

rprouse · 2017-05-24T17:37:46Z

@grokky1 thanks for the info.

Oleksandr-Tokmakov · 2017-06-20T09:46:04Z

using latest 3.8-alpha1 release nunit works with .netcore test project and vs2017 but with .netstandard it doesn't work.
https://github.com/nunit/nunit3-vs-adapter/releases/tag/3.8-alpha1

Your test assemblies must also be .NET Core, not .NET Standard, I don't know why.

can anyone explain what does that mean?
With .net standard test project when I open solution in Tests output I see:

------ Discover test started ------
NUnit Adapter 3.8.0.0: Test discovery starting
Dependent Assembly nunit.framework of ~\Tests\bin\Debug\netstandard1.6\Domain.Tests.dll not found. Can be ignored if not a NUnit project.
NUnit Adapter 3.8.0.0: Test discovery complete
========== Discover test finished: 0 found (0:00:00.5544894) ==========

It looks like minor bug that can be fixed.

AlexeiScherbakov · 2017-06-20T10:18:35Z

net core & net standard is not same. TargetFramework for .net core is netcoreapp1.1 not netstanard1.6 20 Июн 2017 г. 12:46 пользователь "Oleksandr-Tokmakov" < notifications@github.com> написал: using latest 3.8-alpha1 release nunit works with .netcore test project and vs2017 but with .netstandard it doesn't work. https://github.com/nunit/nunit3-vs-adapter/releases/tag/3.8-alpha1 Your test assemblies must also be .NET Core, not .NET Standard, *I don't know why*. can anyone explain what does that mean? With .net standard test project when I open solution in Tests output I see:

…

------ Discover test started ------ NUnit Adapter 3.8.0.0: Test discovery starting Dependent Assembly nunit.framework of ~\Tests\bin\Debug\ netstandard1.6\Domain.Tests.dll not found. Can be ignored if not a NUnit project. NUnit Adapter 3.8.0.0: Test discovery complete ========== Discover test finished: 0 found (0:00:00.5544894) ========== It looks like minor bug that can be fixed. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#305 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AIQ7O30Lyt4rCTMyRNckW26F7kca-TbBks5sF5TegaJpZM4MKjgz> .

OsirisTerje · 2017-10-08T14:28:09Z

I'll close this item now.
The 3.8.0 adapter (>=3.8.0) works with .net core, see http://www.alteridem.net/2017/05/04/test-net-core-nunit-vs2017/ and http://hermit.no/net-core-setup/

@AlexeiScherbakov See posts linked above. .Net standard can be tested, but test project must target a platform.

AlexeiScherbakov · 2017-10-08T14:56:12Z

@OsirisTerje , I think that is not same. My workaround is for people, who installed for Vsix test adapter. And it worked without 'Microsoft.Net.Test.Sdk' package.

In your link 'Trying to use a Vsix test adapter to test .net core. This will not work'. This is issue for VSIX Test Adapter, not for Nuget package, so I think it must stay open before VSIX test adapter can do same thing, or may be before VSIX test adapter is marked as deprecated methods of running tests in VS2017

My test project:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net45</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="NUnit" Version="3.6.1" />
  </ItemGroup>
  <PropertyGroup>
    <DebugType Condition="'$(TargetFramework)' != '' AND '$(TargetFramework)' != 'netcoreapp1.0'">Full</DebugType>
  </PropertyGroup>
  
  <ItemGroup>
    <Reference Include="System.Web" />
  </ItemGroup>
  
  <ItemGroup>
    <Service Include="{82a7f48d-3b50-4b1e-b82e-3ada8210c358}" />
  </ItemGroup>

</Project>

OsirisTerje · 2017-10-08T20:46:44Z

This is unclear to me.
Nowhere earlier in this thread is it mentioned this issue is vsix specific. Do I misunderstand you?

Further, the code for the vsix and the adapter is the same, there is no difference, it is just different packaging, and the nuget adapter works with portable pdb format.
MSTest2 and XUnit do no longer provide vsix packages. (MSTest(1) does not support .net core, but is a vsix.)
Support of vsix is still present, but support for .net core with vsix should not work (from the vstest side, not our), but I may be wrong there.

I have just tested using your workaround too, and it doesnt seem to have any effect. I might be missing something here.

Given new or old csproj format with .net framework, it works, both with the vsix and with the nuget. But, with the .net core 2, it only works with the nuget adapter.

I have asked the MS product group directly if they can help shed some light on this.
But please add more info, if I have misunderstood something here.

sm-g · 2017-10-15T17:39:23Z

Tests from this test project

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net461</TargetFramework>
    <RuntimeIdentifier>win7-x64</RuntimeIdentifier>
    <AssemblyName>X.WebApp.Tests</AssemblyName>
    <PackageId>X.WebApp.Tests</PackageId>
    <PreserveCompilationContext>true</PreserveCompilationContext>
    <CodeAnalysisRuleSet>..\X.Tests.ruleset</CodeAnalysisRuleSet>
  </PropertyGroup>

  <PropertyGroup>
    <AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>
    <GenerateBindingRedirectsOutputType>true</GenerateBindingRedirectsOutputType>
  </PropertyGroup>

  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|AnyCPU'">
    <LangVersion>latest</LangVersion>
  </PropertyGroup>

  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'">
    <LangVersion>latest</LangVersion>
  </PropertyGroup>

  <ItemGroup>
    <ProjectReference Include="..\X.Domain\X.Domain.csproj" />
    <ProjectReference Include="..\X.Tests\X.Tests.csproj" />
    <ProjectReference Include="..\X.WebApp.Models\X.WebApp.Models.csproj" />
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\X.WebApp\X.WebApp.csproj" />
  </ItemGroup>

  <ItemGroup>
    <AdditionalFiles Include="..\stylecop.json" />
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.3.0" />
    <PackageReference Include="AutoFixture.NUnit3" Version="3.50.6" />
    <PackageReference Include="Moq" Version="4.7.99" />
    <PackageReference Include="Newtonsoft.Json" Version="10.0.3" />
    <PackageReference Include="NUnit" Version="3.8.1" />
    <PackageReference Include="Microsoft.DotNet.InternalAbstractions" Version="1.0.500-preview2-1-003177" />
    <PackageReference Include="Microsoft.AspNetCore.TestHost" Version="2.0.0" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.InMemory" Version="2.0.0" />
    <PackageReference Include="NUnit3TestAdapter" Version="3.8.0" />
    <PackageReference Include="SonarAnalyzer.CSharp" Version="6.4.0.3347">
      <PrivateAssets>All</PrivateAssets>
    </PackageReference>
    <PackageReference Include="RefactoringEssentials" Version="5.2.0">
      <PrivateAssets>All</PrivateAssets>
    </PackageReference>
    <PackageReference Include="StyleCop.Analyzers" Version="1.1.0-beta001">
      <PrivateAssets>All</PrivateAssets>
    </PackageReference>
  </ItemGroup>

  <ItemGroup Condition=" '$(TargetFramework)' == 'net461' ">
    <Reference Include="System" />
    <Reference Include="Microsoft.CSharp" />
  </ItemGroup>

  <ItemGroup>
    <Service Include="{82a7f48d-3b50-4b1e-b82e-3ada8210c358}" />
  </ItemGroup>
  <!--
     https://github.com/NuGet/Home/issues/4412
  -->
  <Target Name="CopyDepsFiles" AfterTargets="Build" Condition="'$(TargetFramework)'!=''">
    <ItemGroup>
      <DepsFilePaths Include="$([System.IO.Path]::ChangeExtension('%(_ResolvedProjectReferencePaths.FullPath)', '.deps.json'))" />
    </ItemGroup>

    <Copy SourceFiles="%(DepsFilePaths.FullPath)" DestinationFolder="$(OutputPath)" Condition="Exists('%(DepsFilePaths.FullPath)')" />
  </Target>
</Project>

still not visible on TestExplorer in VS 15.3.5, with or without NUnit 3.8.0 VSIX enabled. Testdriven.net is the only way to run tests from VS I found.

OsirisTerje · 2017-10-15T18:20:43Z

@sm-g Can you please upload or point to a repro project?
And, you have the nuget adapter references, so you should NOT install the vsix at the same time (it will override any nuget adapter, regardless of version)

OsirisTerje · 2017-10-15T19:54:45Z

@sm-g Just also noticed that you're running in x64. Have you switched (from Main VS menu) the Test/Test Settings/Default Architecture to x64 ?

sm-g · 2017-10-16T11:39:02Z

Have you switched (from Main VS menu) the Test/Test Settings/Default Architecture to x64 ?

this was the reason, thanks

rprouse · 2017-10-16T12:21:21Z

Glad you fixed it 👍

Latency · 2018-02-02T05:41:39Z

Doesn't work for netcoreapp2.0

<Project Sdk="Microsoft.NET.Sdk" ToolsVersion="15.0">
  <PropertyGroup>
    <OutputType>Library</OutputType>
    <TargetFramework>netcoreapp2.0</TargetFramework>
  </PropertyGroup>
  <PropertyGroup Condition="'$(Configuration)'=='Debug'">
    <OutputPath>bin\Debug\</OutputPath>
    <DefineConstants>TRACE;DEBUG</DefineConstants>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.5.0" />
    <PackageReference Include="NUnit" Version="3.9.0" />
    <!-- Extension vsix installed for N3Unit Adapter -->
  </ItemGroup>
</Project>

Requires that you replace with.

<TargetFrameworkVersion>v4.7.1</TargetFrameworkVersion>
<TargetFramework>net471</TargetFramework>

Is this a known issue or a bug?

OsirisTerje · 2018-02-02T08:53:55Z

.net core doesnt work with vsix, you must use the nuget adapter

rprouse · 2018-02-06T03:42:49Z

@Latency as @OsirisTerje says, the VSIX version of the test adapter does not work for .NET Core. This is not a limitation of NUnit, but of Visual Studio. Install the NuGet and it will work for you. Sorry, nothing we can do.

CharliePoole mentioned this issue Feb 23, 2017

Visual Studio 2017 RC does not detect nunit tests nunit/nunit#2051

Closed

CharliePoole added confirm We need a separate confirmation of this issue is:bug labels Feb 23, 2017

ChrisMaddock mentioned this issue Mar 14, 2017

Can't run tests in VS2017 (OutOfMemoryException) nunit/nunit-vs-adapter#145

Closed

iancooper mentioned this issue Mar 28, 2017

Support VS 2017 .csproj project format BrighterCommand/Brighter#216

Closed

cesarsouza mentioned this issue Jun 3, 2017

.NET Core version accord-net/framework#318

Closed

OsirisTerje closed this as completed Oct 8, 2017

OsirisTerje added closed:notabug and removed confirm We need a separate confirmation of this issue is:bug labels Oct 21, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Visual Studio 2017 RC does not detect nunit tests #305

Visual Studio 2017 RC does not detect nunit tests #305

CharliePoole commented Feb 23, 2017

claunia commented Feb 23, 2017

CharliePoole commented Feb 23, 2017

ChrisMaddock commented Feb 24, 2017

CharliePoole commented Feb 24, 2017

NZSmartie commented Feb 26, 2017

NZSmartie commented Feb 26, 2017

qqqwwweeea commented Mar 4, 2017

AlexeiScherbakov commented Mar 8, 2017

JessePreiner commented Mar 29, 2017 •

edited

jhickson commented Mar 31, 2017

qrli commented Apr 4, 2017

grokky1 commented May 24, 2017 •

edited

jnm2 commented May 24, 2017

rprouse commented May 24, 2017

grokky1 commented May 24, 2017 •

edited

rprouse commented May 24, 2017

Oleksandr-Tokmakov commented Jun 20, 2017

AlexeiScherbakov commented Jun 20, 2017 via email

OsirisTerje commented Oct 8, 2017

AlexeiScherbakov commented Oct 8, 2017

OsirisTerje commented Oct 8, 2017

sm-g commented Oct 15, 2017

OsirisTerje commented Oct 15, 2017 •

edited

OsirisTerje commented Oct 15, 2017

sm-g commented Oct 16, 2017

rprouse commented Oct 16, 2017

Latency commented Feb 2, 2018

OsirisTerje commented Feb 2, 2018

rprouse commented Feb 6, 2018

Visual Studio 2017 RC does not detect nunit tests #305

Visual Studio 2017 RC does not detect nunit tests #305

Comments

CharliePoole commented Feb 23, 2017

claunia commented Feb 23, 2017

CharliePoole commented Feb 23, 2017

ChrisMaddock commented Feb 24, 2017

CharliePoole commented Feb 24, 2017

NZSmartie commented Feb 26, 2017

NZSmartie commented Feb 26, 2017

qqqwwweeea commented Mar 4, 2017

AlexeiScherbakov commented Mar 8, 2017

JessePreiner commented Mar 29, 2017 • edited

jhickson commented Mar 31, 2017

qrli commented Apr 4, 2017

grokky1 commented May 24, 2017 • edited

jnm2 commented May 24, 2017

rprouse commented May 24, 2017

grokky1 commented May 24, 2017 • edited

rprouse commented May 24, 2017

Oleksandr-Tokmakov commented Jun 20, 2017

AlexeiScherbakov commented Jun 20, 2017 via email

OsirisTerje commented Oct 8, 2017

AlexeiScherbakov commented Oct 8, 2017

OsirisTerje commented Oct 8, 2017

sm-g commented Oct 15, 2017

OsirisTerje commented Oct 15, 2017 • edited

OsirisTerje commented Oct 15, 2017

sm-g commented Oct 16, 2017

rprouse commented Oct 16, 2017

Latency commented Feb 2, 2018

OsirisTerje commented Feb 2, 2018

rprouse commented Feb 6, 2018

JessePreiner commented Mar 29, 2017 •

edited

grokky1 commented May 24, 2017 •

edited

grokky1 commented May 24, 2017 •

edited

OsirisTerje commented Oct 15, 2017 •

edited