New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for .Net Core projects #297

Closed
CharliePoole opened this Issue Jan 14, 2017 · 76 comments

Comments

Projects
None yet
@CharliePoole
Member

CharliePoole commented Jan 14, 2017

@galvesribeiro commented on Sat Jan 14 2017

Visual Studio is throwing this:

------ Discover test started ------
NUnit Adapter 3.6.0.0: Test discovery starting
Error: Unable to get runner for this assembly. Check installation, including any extensions.
FileNotFoundException: Could not load file or assembly 'nunit.framework' or one of its dependencies. The system cannot find the file specified.
Dependent Assembly nunit.framework of C:\Users\gutem\documents\visual studio 2017\Projects\ClassLibrary1\ClassLibrary1\bin\Debug\netstandard1.6\ClassLibrary1.dll not found. Can be ignored if not a NUnit project.
NUnit Adapter 3.6.0.0: Test discovery complete
========== Discover test finished: 0 found (0:00:00.655) ==========

This is the sample .csproj:

<Project Sdk="Microsoft.NET.Sdk" ToolsVersion="15.0">
  <PropertyGroup>
    <TargetFramework>netstandard1.6</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <Compile Include="**\*.cs" />
    <EmbeddedResource Include="**\*.resx" />
  </ItemGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.0.0-preview-20170113-02" />
    <PackageReference Include="NETStandard.Library" Version="1.6.1" />
    <PackageReference Include="NUnit" Version="3.6.0" />
    <PackageReference Include="NUnit3TestAdapter" Version="3.6.0" />
  </ItemGroup>
</Project>

With this sample test:

[TestFixture]
    public class Class1
    {
        string teststring = "";

        [SetUp]
        public void SetUp()
        {
            teststring = "AAA";
        }

        [Test]
        public void TestDemo()
        {
            Assert.True(teststring.Equals("AAA"));
        }
    }

Do we need to do anything else?

Thank you.


@jnm2 commented on Sat Jan 14 2017

@CharliePoole I wonder if this is #296?

@galvesribeiro Can you see if the NUnit console runner runs the tests properly?


@galvesribeiro commented on Sat Jan 14 2017

@jnm2 what should I do to use the console runner on .net standard projects?


@JustinRChou commented on Sat Jan 14 2017

Seems like it.
NUnit Adapter 3.6.1.0 has the same issue.


@jnm2 commented on Sat Jan 14 2017

@galvesribeiro Download the zip from https://github.com/nunit/nunit-console/releases and run nunit3-console "path\to\test\assembly.dll" at the command line.


@jnm2 commented on Sat Jan 14 2017

@JustinRChou can we make these new-style .csproj's part of the VS adapter tests, both .NET Standard and .NET Framework? A single project can multitarget and produce both binaries.

@rprouse

This comment has been minimized.

Show comment
Hide comment
@rprouse

rprouse Jan 14, 2017

Member

@galvesribeiro this is a known issue. The NUnit Engine does not currently support .NET Core projects and the dotnet-test-nunit adapter doesn't work with the new .NET Core CSPROJ format. Microsoft decided to drop support for the various dotnet-test-* style adapters in favor of the old style visual studio adapters and we are still playing catchup.

Member

rprouse commented Jan 14, 2017

@galvesribeiro this is a known issue. The NUnit Engine does not currently support .NET Core projects and the dotnet-test-nunit adapter doesn't work with the new .NET Core CSPROJ format. Microsoft decided to drop support for the various dotnet-test-* style adapters in favor of the old style visual studio adapters and we are still playing catchup.

@galvesribeiro

This comment has been minimized.

Show comment
Hide comment
@galvesribeiro

galvesribeiro Jan 14, 2017

@rprouse that is strange... xUnit still working just fine since the beginning of .Net Core with DNX. I though when you released 3.6 which is supposed to support .Net Standard 1.6 projects it should work. My mistake...

@rprouse that is strange... xUnit still working just fine since the beginning of .Net Core with DNX. I though when you released 3.6 which is supposed to support .Net Standard 1.6 projects it should work. My mistake...

@rprouse

This comment has been minimized.

Show comment
Hide comment
@rprouse

rprouse Jan 14, 2017

Member

@galvesribeiro xUnit has the advantage that the .NET core team is using xUnit, so they were updated first 😄

We released the .NET Standard version of the framework as a first step towards runner support. As a holdover until we are ready, you can make your tests self-executable using NUnitLite.

Member

rprouse commented Jan 14, 2017

@galvesribeiro xUnit has the advantage that the .NET core team is using xUnit, so they were updated first 😄

We released the .NET Standard version of the framework as a first step towards runner support. As a holdover until we are ready, you can make your tests self-executable using NUnitLite.

@galvesribeiro

This comment has been minimized.

Show comment
Hide comment
@galvesribeiro

galvesribeiro Jan 14, 2017

@rprouse right. The only problem with that is that I can't use VS to run/debug the tests... Ok, I understood...

Thanks, please close the issue.

@rprouse right. The only problem with that is that I can't use VS to run/debug the tests... Ok, I understood...

Thanks, please close the issue.

@jnm2

This comment has been minimized.

Show comment
Hide comment
@jnm2

jnm2 Jan 14, 2017

Contributor

@rprouse Are you planning to support .NET Standard 1.4 or 2.0? The reason I ask is that I have .NET Standard DLLs that I want to test on the .NET Framework, not .NET Core, runtime.

Contributor

jnm2 commented Jan 14, 2017

@rprouse Are you planning to support .NET Standard 1.4 or 2.0? The reason I ask is that I have .NET Standard DLLs that I want to test on the .NET Framework, not .NET Core, runtime.

@rprouse

This comment has been minimized.

Show comment
Hide comment
@rprouse

rprouse Jan 14, 2017

Member

@jnm2 we will probably replace the PCL project with .NET Standard 1.0 or 1.3 and we may add 2.0 support if needed. Remember, you can always use a lower version of .NET Standard NUnit framework in a higher level project.

Member

rprouse commented Jan 14, 2017

@jnm2 we will probably replace the PCL project with .NET Standard 1.0 or 1.3 and we may add 2.0 support if needed. Remember, you can always use a lower version of .NET Standard NUnit framework in a higher level project.

@CharliePoole

This comment has been minimized.

Show comment
Hide comment
@CharliePoole

CharliePoole Jan 14, 2017

Member

Lost the internet for a while or I would have commented sooner...

The 3.6 release of the adapter is not the new one we are working on. It came out a few months ago, before we had any support for .NET Standard. I'm working on finishing the 3.7 release, but that isn't going to solve the problem either.

Runners (like the console, gui or VS adapter) depend on the nunit engine to load and run tests. As soon as we have an engine that supports .NET standard, we'll be able to make the runners support it. Since we release the console together with the engine, that support will be immediate. For the adapter, it won't happen until (a) there is an engine with support and (b) the adapter is updated to use that engine.

We won't close this because this is support we want to provide. But I'm marking it as blocked until there is engine support for it.

Member

CharliePoole commented Jan 14, 2017

Lost the internet for a while or I would have commented sooner...

The 3.6 release of the adapter is not the new one we are working on. It came out a few months ago, before we had any support for .NET Standard. I'm working on finishing the 3.7 release, but that isn't going to solve the problem either.

Runners (like the console, gui or VS adapter) depend on the nunit engine to load and run tests. As soon as we have an engine that supports .NET standard, we'll be able to make the runners support it. Since we release the console together with the engine, that support will be immediate. For the adapter, it won't happen until (a) there is an engine with support and (b) the adapter is updated to use that engine.

We won't close this because this is support we want to provide. But I'm marking it as blocked until there is engine support for it.

@CharliePoole CharliePoole changed the title from NUnit 3.6.0 don't find tests on .Net Standard 1.6 projects to Support for .Net Standard 1.6 projects Jan 14, 2017

@jnm2

This comment has been minimized.

Show comment
Hide comment
@jnm2

jnm2 Jan 14, 2017

Contributor

@CharliePoole when you have a project that targets netcoreapp10;net45;net46 it's obvious what to do- run each of the three output binaries against its target TFM. (The reason you might want to test against both 4.5 and 4.6 is quirking.)

However, if one of a project's targets is netstandard14, what runtime is that output binary tested against? .NET Core and .NET Framework are both common candidates and there are other runtimes besides. It would be awesome if I could compile to a single .NET Standard binary and do all my testing off that same binary for every platform (.NET Core, .NET Framework, Xamarin, UWP, whatever).

Will there be a way to specify to NUnit which runtimes to test against, or will it test against all of them that it can?

Contributor

jnm2 commented Jan 14, 2017

@CharliePoole when you have a project that targets netcoreapp10;net45;net46 it's obvious what to do- run each of the three output binaries against its target TFM. (The reason you might want to test against both 4.5 and 4.6 is quirking.)

However, if one of a project's targets is netstandard14, what runtime is that output binary tested against? .NET Core and .NET Framework are both common candidates and there are other runtimes besides. It would be awesome if I could compile to a single .NET Standard binary and do all my testing off that same binary for every platform (.NET Core, .NET Framework, Xamarin, UWP, whatever).

Will there be a way to specify to NUnit which runtimes to test against, or will it test against all of them that it can?

@CharliePoole

This comment has been minimized.

Show comment
Hide comment
@CharliePoole

CharliePoole Jan 15, 2017

Member

@jnm2 It's less obvious how to do it. 😄

We are a project that needs to test against multiple runtimes and we do it by replicating test projects. It would be cool to be able to do it in an easier way, but first we have to be able to run the net standard tests at all. IOW, I'm trying to figure out how to get into orbit and you're ready to land on Mars. 😄

NUnit - up to now - makes no decisions on this. It just runs against what you tell it to run against. I don't think that automatically running against multiple runtimes can be an engine feature. It has to be in the runner. The reason I say that is that runners are likely to get pretty confused if they tell the engine to run an assembly and it runs it three times. The runner has to be ready to receive what the engine produces. OTOH, the engine could introduce a setting that allows passing multiple platforms as we now pass in a single platform.

Currently, if you have a .NET 2.0 assembly (to use old technology as an example) you can tell the engine to run it under .NET 4.0. If you don't tell it, it will use 2.0 if available, otherwise the lowest compatible runtime available. Hypothetically, we could pass in an argument something like "net-2.0+net-4.0+net-4.5" and expect it to be run three times. Something similar could be done for portable or net standard tests - it's just that nobody ever thought of it before. Feel free to introduce it as an issue so we don't forget it and also to keep this issue clean.

Member

CharliePoole commented Jan 15, 2017

@jnm2 It's less obvious how to do it. 😄

We are a project that needs to test against multiple runtimes and we do it by replicating test projects. It would be cool to be able to do it in an easier way, but first we have to be able to run the net standard tests at all. IOW, I'm trying to figure out how to get into orbit and you're ready to land on Mars. 😄

NUnit - up to now - makes no decisions on this. It just runs against what you tell it to run against. I don't think that automatically running against multiple runtimes can be an engine feature. It has to be in the runner. The reason I say that is that runners are likely to get pretty confused if they tell the engine to run an assembly and it runs it three times. The runner has to be ready to receive what the engine produces. OTOH, the engine could introduce a setting that allows passing multiple platforms as we now pass in a single platform.

Currently, if you have a .NET 2.0 assembly (to use old technology as an example) you can tell the engine to run it under .NET 4.0. If you don't tell it, it will use 2.0 if available, otherwise the lowest compatible runtime available. Hypothetically, we could pass in an argument something like "net-2.0+net-4.0+net-4.5" and expect it to be run three times. Something similar could be done for portable or net standard tests - it's just that nobody ever thought of it before. Feel free to introduce it as an issue so we don't forget it and also to keep this issue clean.

@galvesribeiro

This comment has been minimized.

Show comment
Hide comment
@galvesribeiro

galvesribeiro Jan 15, 2017

I'm not sure if run multiple times against multiple frameworks is something that worth... The principle of netstandard is to support the same API surface between multiple TFMs so, I believe if you target your library to netstandard even if your main facus are net45 framework would be the best solution. Multitargeting on netstandard tooling is supported but is not encouraged since it goes against its purpose.

I'm not sure if run multiple times against multiple frameworks is something that worth... The principle of netstandard is to support the same API surface between multiple TFMs so, I believe if you target your library to netstandard even if your main facus are net45 framework would be the best solution. Multitargeting on netstandard tooling is supported but is not encouraged since it goes against its purpose.

@galvesribeiro

This comment has been minimized.

Show comment
Hide comment
@galvesribeiro

galvesribeiro Jan 15, 2017

But like @CharliePoole said, if you REALLY want to test it that way, make your library to multitarget and create multiple test projects and if they are all have the same set of test code, put it in a shared test project which is referenced on each test project for that specific platform.

But like @CharliePoole said, if you REALLY want to test it that way, make your library to multitarget and create multiple test projects and if they are all have the same set of test code, put it in a shared test project which is referenced on each test project for that specific platform.

@jnm2

This comment has been minimized.

Show comment
Hide comment
@jnm2

jnm2 Jan 15, 2017

Contributor

@galvesribeiro The way I really want to test is to produce a single netstandard output and have it run against various versions of .NET Core and .NET Framework. I don't want to multitarget if I don't have to. Then again, there will be scenarios where I need to target netstandard1.4;net40 and I won't have an option.

Contributor

jnm2 commented Jan 15, 2017

@galvesribeiro The way I really want to test is to produce a single netstandard output and have it run against various versions of .NET Core and .NET Framework. I don't want to multitarget if I don't have to. Then again, there will be scenarios where I need to target netstandard1.4;net40 and I won't have an option.

@jnm2

This comment has been minimized.

Show comment
Hide comment
@jnm2

jnm2 Jan 15, 2017

Contributor

@CharliePoole

Feel free to introduce it as an issue so we don't forget it and also to keep this issue clean.

I'm happy to. Not sure where to put the issue since it affects more than one runner.

Let's just take a common case, where I'm forced to target netstandard14;net40. Which runtimes will be used to test the two binaries? What would be ideal for me is for the net40 output binary to be tested against net40 and for the netstandard14 output binary to be tested against the earliest version of .NET Core (1.0) and the earliest version of .NET Framework (4.6.1) that support it. However, someone else may want to also test the netstandard binary against UWP 10 or Xamarin vNext or the latest .NET Core.

Contributor

jnm2 commented Jan 15, 2017

@CharliePoole

Feel free to introduce it as an issue so we don't forget it and also to keep this issue clean.

I'm happy to. Not sure where to put the issue since it affects more than one runner.

Let's just take a common case, where I'm forced to target netstandard14;net40. Which runtimes will be used to test the two binaries? What would be ideal for me is for the net40 output binary to be tested against net40 and for the netstandard14 output binary to be tested against the earliest version of .NET Core (1.0) and the earliest version of .NET Framework (4.6.1) that support it. However, someone else may want to also test the netstandard binary against UWP 10 or Xamarin vNext or the latest .NET Core.

@CharliePoole

This comment has been minimized.

Show comment
Hide comment
@CharliePoole

CharliePoole Jan 15, 2017

Member

@jnm2 I think it has to be done in the engine and console first. It can be added elsewhere after that's done. Each runner would get a separate issue because we are trying to treat each runner as a separate project but there's no point in creating (for example) an adapter issue until there's a feature in the engine that allows the adapter to do this.

Currently the engine runs tests once, under the specified framework or an automatically selected framework. Since the engine doesn't work with multi-targeted assemblies, it can always automatically select a single framework. I think the initial option needed to power all of this is the ability to tell the engine to run tests under multiple frameworks. Everything comes from that.

Member

CharliePoole commented Jan 15, 2017

@jnm2 I think it has to be done in the engine and console first. It can be added elsewhere after that's done. Each runner would get a separate issue because we are trying to treat each runner as a separate project but there's no point in creating (for example) an adapter issue until there's a feature in the engine that allows the adapter to do this.

Currently the engine runs tests once, under the specified framework or an automatically selected framework. Since the engine doesn't work with multi-targeted assemblies, it can always automatically select a single framework. I think the initial option needed to power all of this is the ability to tell the engine to run tests under multiple frameworks. Everything comes from that.

@jnm2

This comment has been minimized.

Show comment
Hide comment
@jnm2

jnm2 Jan 15, 2017

Contributor

I understood you to be saying it would not be done in the engine:

I don't think that automatically running against multiple runtimes can be an engine feature. It has to be in the runner. The reason I say that is that runners are likely to get pretty confused if they tell the engine to run an assembly and it runs it three times.

But since I quietly disagreed with that earlier, I'll just stay out of the way now 😆

Contributor

jnm2 commented Jan 15, 2017

I understood you to be saying it would not be done in the engine:

I don't think that automatically running against multiple runtimes can be an engine feature. It has to be in the runner. The reason I say that is that runners are likely to get pretty confused if they tell the engine to run an assembly and it runs it three times.

But since I quietly disagreed with that earlier, I'll just stay out of the way now 😆

@CharliePoole

This comment has been minimized.

Show comment
Hide comment
@CharliePoole

CharliePoole Jan 15, 2017

Member

If we want the tests to run three times in one Run call, then the engine has to do that. What I wrote earlier was that the engine can't do that automatically because it doesn't know whether the runner calling it can handle the triple output. The engine isn't intended to put demands on runners, but to accept commands from them.

OTOH, in order for a runner to command the engine to do it, we have to implement the capability in the engine. Does that make more sense?

Member

CharliePoole commented Jan 15, 2017

If we want the tests to run three times in one Run call, then the engine has to do that. What I wrote earlier was that the engine can't do that automatically because it doesn't know whether the runner calling it can handle the triple output. The engine isn't intended to put demands on runners, but to accept commands from them.

OTOH, in order for a runner to command the engine to do it, we have to implement the capability in the engine. Does that make more sense?

@CharliePoole

This comment has been minimized.

Show comment
Hide comment
@CharliePoole

CharliePoole Jan 15, 2017

Member

BTW,,, I missed your earlier disagreement... still can't find it.

Member

CharliePoole commented Jan 15, 2017

BTW,,, I missed your earlier disagreement... still can't find it.

@jnm2

This comment has been minimized.

Show comment
Hide comment
@jnm2

jnm2 Jan 15, 2017

Contributor

That makes perfect sense.

BTW,,, I missed your earlier disagreement... still can't find it.

quietly internally.
Everything looks good. I'll try not to confuse anything from this point forward. 🙄

Contributor

jnm2 commented Jan 15, 2017

That makes perfect sense.

BTW,,, I missed your earlier disagreement... still can't find it.

quietly internally.
Everything looks good. I'll try not to confuse anything from this point forward. 🙄

@rprouse

This comment has been minimized.

Show comment
Hide comment
@rprouse

rprouse Jan 25, 2017

Member

Now that VSTest has been open sourced, I have been reading through the documentation and wanted to add some notes to this issue in preparation for working on it. This issue is blocked until we have .NET Core support in the engine, but how we implement that support in the engine may be driven by how we will consume it in this adapter.

For example, AFAIK, this adapter runs tests in-process and does not use the agents. The new dotnet test command line will also use this adapter to run tests at the command line. That will be the documented way to run tests from the command line, so do we need to also add support to the console runner?

I've been looking at how MSTest and xUnit package their adapters now to support .NET Core. It is very similar to other NuGet packages with platform specific versions of the *.TestAdapter.dll in platform directories. Both place their adapters in a Build subdirectory, not a Tools directory. Another interesting thing is the props file which points to platform independent versions of referenced assemblies.

Here is the layout for the MSTest adapter,

image

Notice that the Microsoft.VisualStudio.TestPlatform.TestFramework.dll is in the _common directory and only platform specific assemblies are in the platform directories. The test framework is .NET standard.

The props files in each platform directory then tell it which assemblies to use for each platform target.

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <ItemGroup>
    <Content Include="$(MSBuildThisFileDirectory)..\_common\Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.dll">
      <Link>Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.dll</Link>
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
      <Visible>False</Visible>
    </Content>
    <Content Include="$(MSBuildThisFileDirectory)..\_common\Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.Interface.dll">
      <Link>Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.Interface.dll</Link>
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
      <Visible>False</Visible>
    </Content>
	<Content Include="$(MSBuildThisFileDirectory)..\uap10.0\Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.dll">
      <Link>Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.dll</Link>
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
      <Visible>False</Visible>
    </Content>
  </ItemGroup>
</Project>

I am not sure what the targets files are for, fallback resource files for translations?

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    <EnableMSTestV2CopyResources Condition="$(EnableMSTestV2CopyResources) == ''">true</EnableMSTestV2CopyResources>
  </PropertyGroup>

  <Target Name="GetMSTestV2CultureHierarchy">
    <!-- Only traversing 5 levels in the culture hierarchy. This is the maximum lenght for all cultures and should be sufficient to get to a culture name that maps to a resource folder we package. 
    The root culture name for all cultures is invariant whose name is ''(empty) and the parent for invariant culture is invariant itself.(https://msdn.microsoft.com/en-us/library/system.globalization.cultureinfo.parent(v=vs.110).aspx.) 
    So the below code should not break build in any case. -->
    <ItemGroup>
      <CurrentUICultureHierarchy Include="$([System.Globalization.CultureInfo]::CurrentUICulture.Name)" />
      <CurrentUICultureHierarchy Include="$([System.Globalization.CultureInfo]::CurrentUICulture.Parent.Name)"  Condition="$([System.Globalization.CultureInfo]::CurrentUICulture.Parent.Name) != ''"/>
      <CurrentUICultureHierarchy Include="$([System.Globalization.CultureInfo]::CurrentUICulture.Parent.Parent.Name)"  Condition="$([System.Globalization.CultureInfo]::CurrentUICulture.Parent.Parent.Name) != ''"/>
      <CurrentUICultureHierarchy Include="$([System.Globalization.CultureInfo]::CurrentUICulture.Parent.Parent.Parent.Name)"  Condition="$([System.Globalization.CultureInfo]::CurrentUICulture.Parent.Parent.Parent.Name) != ''"/>
      <CurrentUICultureHierarchy Include="$([System.Globalization.CultureInfo]::CurrentUICulture.Parent.Parent.Parent.Parent.Name)"  Condition="$([System.Globalization.CultureInfo]::CurrentUICulture.Parent.Parent.Parent.Parent.Name) != ''"/>
    </ItemGroup>
  </Target>

  <!-- Copy resources over to $(TargetDir) if this is a localized build. -->
  <Target Name="CopyMSTestV2Resources" BeforeTargets="PrepareForBuild" Condition="$(EnableMSTestV2CopyResources) == 'true'" DependsOnTargets="GetMSTestV2CultureHierarchy">
    <ItemGroup>
      <MSTestV2ResourceFiles Include="$(MSBuildThisFileDirectory)..\_common\%(CurrentUICultureHierarchy.Identity)\*resources.dll">
        <CultureString>%(CurrentUICultureHierarchy.Identity)</CultureString>
      </MSTestV2ResourceFiles>

      <Content Include="@(MSTestV2ResourceFiles)" Condition="@(MSTestV2ResourceFiles) != ''">
        <Link>%(MSTestV2ResourceFiles.CultureString)\%(Filename)%(Extension)</Link>
        <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
        <Visible>False</Visible>
      </Content>
    </ItemGroup>
  </Target>

  <!-- This is required because an empty resource folder is left even though the files within are cleaned up. -->
  <Target Name="CleanupMSTestV2ResourceFolders" AfterTargets="AfterClean" Condition="$(EnableMSTestV2CopyResources) == 'true'" DependsOnTargets="GetMSTestV2CultureHierarchy">
    <ItemGroup>
       <ResourceDirectories Include="$(TargetDir)%(CurrentUICultureHierarchy.Identity)" />
    </ItemGroup>
    <!-- RemoveDir does not throw if the folder does not exist. Continue on error - In any case do not fail build if this task fails(Warn and move on).-->
    <RemoveDir Directories="@(ResourceDirectories)" ContinueOnError="true"/>
  </Target>
</Project>
Member

rprouse commented Jan 25, 2017

Now that VSTest has been open sourced, I have been reading through the documentation and wanted to add some notes to this issue in preparation for working on it. This issue is blocked until we have .NET Core support in the engine, but how we implement that support in the engine may be driven by how we will consume it in this adapter.

For example, AFAIK, this adapter runs tests in-process and does not use the agents. The new dotnet test command line will also use this adapter to run tests at the command line. That will be the documented way to run tests from the command line, so do we need to also add support to the console runner?

I've been looking at how MSTest and xUnit package their adapters now to support .NET Core. It is very similar to other NuGet packages with platform specific versions of the *.TestAdapter.dll in platform directories. Both place their adapters in a Build subdirectory, not a Tools directory. Another interesting thing is the props file which points to platform independent versions of referenced assemblies.

Here is the layout for the MSTest adapter,

image

Notice that the Microsoft.VisualStudio.TestPlatform.TestFramework.dll is in the _common directory and only platform specific assemblies are in the platform directories. The test framework is .NET standard.

The props files in each platform directory then tell it which assemblies to use for each platform target.

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <ItemGroup>
    <Content Include="$(MSBuildThisFileDirectory)..\_common\Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.dll">
      <Link>Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.dll</Link>
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
      <Visible>False</Visible>
    </Content>
    <Content Include="$(MSBuildThisFileDirectory)..\_common\Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.Interface.dll">
      <Link>Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.Interface.dll</Link>
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
      <Visible>False</Visible>
    </Content>
	<Content Include="$(MSBuildThisFileDirectory)..\uap10.0\Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.dll">
      <Link>Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.dll</Link>
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
      <Visible>False</Visible>
    </Content>
  </ItemGroup>
</Project>

I am not sure what the targets files are for, fallback resource files for translations?

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    <EnableMSTestV2CopyResources Condition="$(EnableMSTestV2CopyResources) == ''">true</EnableMSTestV2CopyResources>
  </PropertyGroup>

  <Target Name="GetMSTestV2CultureHierarchy">
    <!-- Only traversing 5 levels in the culture hierarchy. This is the maximum lenght for all cultures and should be sufficient to get to a culture name that maps to a resource folder we package. 
    The root culture name for all cultures is invariant whose name is ''(empty) and the parent for invariant culture is invariant itself.(https://msdn.microsoft.com/en-us/library/system.globalization.cultureinfo.parent(v=vs.110).aspx.) 
    So the below code should not break build in any case. -->
    <ItemGroup>
      <CurrentUICultureHierarchy Include="$([System.Globalization.CultureInfo]::CurrentUICulture.Name)" />
      <CurrentUICultureHierarchy Include="$([System.Globalization.CultureInfo]::CurrentUICulture.Parent.Name)"  Condition="$([System.Globalization.CultureInfo]::CurrentUICulture.Parent.Name) != ''"/>
      <CurrentUICultureHierarchy Include="$([System.Globalization.CultureInfo]::CurrentUICulture.Parent.Parent.Name)"  Condition="$([System.Globalization.CultureInfo]::CurrentUICulture.Parent.Parent.Name) != ''"/>
      <CurrentUICultureHierarchy Include="$([System.Globalization.CultureInfo]::CurrentUICulture.Parent.Parent.Parent.Name)"  Condition="$([System.Globalization.CultureInfo]::CurrentUICulture.Parent.Parent.Parent.Name) != ''"/>
      <CurrentUICultureHierarchy Include="$([System.Globalization.CultureInfo]::CurrentUICulture.Parent.Parent.Parent.Parent.Name)"  Condition="$([System.Globalization.CultureInfo]::CurrentUICulture.Parent.Parent.Parent.Parent.Name) != ''"/>
    </ItemGroup>
  </Target>

  <!-- Copy resources over to $(TargetDir) if this is a localized build. -->
  <Target Name="CopyMSTestV2Resources" BeforeTargets="PrepareForBuild" Condition="$(EnableMSTestV2CopyResources) == 'true'" DependsOnTargets="GetMSTestV2CultureHierarchy">
    <ItemGroup>
      <MSTestV2ResourceFiles Include="$(MSBuildThisFileDirectory)..\_common\%(CurrentUICultureHierarchy.Identity)\*resources.dll">
        <CultureString>%(CurrentUICultureHierarchy.Identity)</CultureString>
      </MSTestV2ResourceFiles>

      <Content Include="@(MSTestV2ResourceFiles)" Condition="@(MSTestV2ResourceFiles) != ''">
        <Link>%(MSTestV2ResourceFiles.CultureString)\%(Filename)%(Extension)</Link>
        <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
        <Visible>False</Visible>
      </Content>
    </ItemGroup>
  </Target>

  <!-- This is required because an empty resource folder is left even though the files within are cleaned up. -->
  <Target Name="CleanupMSTestV2ResourceFolders" AfterTargets="AfterClean" Condition="$(EnableMSTestV2CopyResources) == 'true'" DependsOnTargets="GetMSTestV2CultureHierarchy">
    <ItemGroup>
       <ResourceDirectories Include="$(TargetDir)%(CurrentUICultureHierarchy.Identity)" />
    </ItemGroup>
    <!-- RemoveDir does not throw if the folder does not exist. Continue on error - In any case do not fail build if this task fails(Warn and move on).-->
    <RemoveDir Directories="@(ResourceDirectories)" ContinueOnError="true"/>
  </Target>
</Project>
@rprouse

This comment has been minimized.

Show comment
Hide comment
@rprouse

rprouse Jan 25, 2017

Member

Here is the layout of the xUnit adapter,

image

Member

rprouse commented Jan 25, 2017

Here is the layout of the xUnit adapter,

image

@galvesribeiro

This comment has been minimized.

Show comment
Hide comment
@galvesribeiro

galvesribeiro Jan 25, 2017

@rprouse the new SDK removes a lot of boilerplate code from the .csproj because it automatically import/infer some props/targets from the .Net SDK. Those props are targeting the correct reference for each one of the underlying multitargeted platforms.

I would suggest take that approach for the Engine and any dependency library. The test adapter can be a netstandard library. Don't need to multitarget I guess.

Regarding the console... There are 2 ways to do it AFAIK. (1) Use the test adapter SDK as xUnit guys did and (2) create a Tool which I don't think is well documented yet on .Net SDK. The tool can be used while building the project or as a standalone exe. But I would not be concerned about it. Everyone targeting netstandard (even people on net4xx) will eventually fall in dotnet CLI, so they will end up using dotnet test from now on. If people stay behind not using dotnet CLI, they can just use the old test mode/nugets/runner IMHO.

@rprouse the new SDK removes a lot of boilerplate code from the .csproj because it automatically import/infer some props/targets from the .Net SDK. Those props are targeting the correct reference for each one of the underlying multitargeted platforms.

I would suggest take that approach for the Engine and any dependency library. The test adapter can be a netstandard library. Don't need to multitarget I guess.

Regarding the console... There are 2 ways to do it AFAIK. (1) Use the test adapter SDK as xUnit guys did and (2) create a Tool which I don't think is well documented yet on .Net SDK. The tool can be used while building the project or as a standalone exe. But I would not be concerned about it. Everyone targeting netstandard (even people on net4xx) will eventually fall in dotnet CLI, so they will end up using dotnet test from now on. If people stay behind not using dotnet CLI, they can just use the old test mode/nugets/runner IMHO.

@codito

This comment has been minimized.

Show comment
Hide comment
@codito

codito Feb 11, 2017

@CharliePoole @jnm2 the dotnet test runner already looks into msbuild to find the target frameworks, accordingly it loads up the test adapters within corresponding runtime context (testhost.exe for net46, dotnet testhost.dll for netcoreapp). As long as the nunit engine and adapter are available for net46, netcoreapp1.0, test runs will just work :)

@jnm2 if a library is netstandard14;net46 and needs to be tested on UWP, NETCore and NET46; user can multitarget the test project only to uap10.0;netcoreapp1.0;net46. dotnet test takes over from there. netstandard14 build of product library is automatically tested in uap10.0 and netcoreapp1.0 runtime contexts. To test a single target framework, user can do dotnet test -f netcoreapp1.0.

Hope it clarifies the use case a bit :)

@rprouse both mstest and xunit leverage nuget's build extensibility to copy over testadapters to output directory of test project. dotnet test tries to find and load possible test adapters (*.TestAdapter.dll regex) from test project's dependency closure. In VS 2017, vstest.console.exe also tries to load test adapters from output directory (default if /TestAdapterPath command line switch is not specified).

MSTest does include an extra target to copy localized resources.

codito commented Feb 11, 2017

@CharliePoole @jnm2 the dotnet test runner already looks into msbuild to find the target frameworks, accordingly it loads up the test adapters within corresponding runtime context (testhost.exe for net46, dotnet testhost.dll for netcoreapp). As long as the nunit engine and adapter are available for net46, netcoreapp1.0, test runs will just work :)

@jnm2 if a library is netstandard14;net46 and needs to be tested on UWP, NETCore and NET46; user can multitarget the test project only to uap10.0;netcoreapp1.0;net46. dotnet test takes over from there. netstandard14 build of product library is automatically tested in uap10.0 and netcoreapp1.0 runtime contexts. To test a single target framework, user can do dotnet test -f netcoreapp1.0.

Hope it clarifies the use case a bit :)

@rprouse both mstest and xunit leverage nuget's build extensibility to copy over testadapters to output directory of test project. dotnet test tries to find and load possible test adapters (*.TestAdapter.dll regex) from test project's dependency closure. In VS 2017, vstest.console.exe also tries to load test adapters from output directory (default if /TestAdapterPath command line switch is not specified).

MSTest does include an extra target to copy localized resources.

@jnm2

This comment has been minimized.

Show comment
Hide comment
@jnm2

jnm2 Feb 11, 2017

Contributor

@codito Thank you! I'm guessing this is a follow up from dotnet/sdk#833 (comment)?

If I understand correctly, the current guidance is to tell people not to target netstandard at all in the top-level test projects being executed, but rather multitarget them to specific platforms and versions like you're saying. This gives the test runner a DLL per platform and version and the test runner should be able to tell which platform and version to use by examining each DLL.

Here's a question I'm struggling with: if the NUnit framework projects take advantage of multitargeting, they'll probably consolidate to a single project targeting net20;net35;net40;net45;netstandard1.6;.NETPortable,Version=v4.5,Profile=Profile259.

What should the framework test projects should be targeting? Due to preprocessor conditions, each of these six framework targets runs slightly different code.

That means we'd want to:

  • run tests on the net20 framework DLL on net20
  • run tests on the net35 framework DLL on net35
  • run tests on the net40 framework DLL on net40
  • run tests on the net45 framework DLL on net45
  • run tests on the netstandard1.6 framework DLL on netcore1.0
  • run tests on the netstandard1.6 framework DLL on net47 (maybe)
  • run tests on the .NETPortable,Version=v4.5,Profile=Profile259 framework DLL on net45
  • run tests on the .NETPortable,Version=v4.5,Profile=Profile259 framework DLL on win8 or netcore1.0 (maybe)

In particular, the test projects need to end up running tests against net45 twice: once for the net45 framework DLL, and once for the portable framework DLL.

The simple transformation net20;net35;net40;net45;netstandard1.6;.NETPortable,Version=v4.5,Profile=Profile259 -> net20;net35;net40;net45;netcoreapp1.0;net45 doesn't really make sense anymore. What's a good way to handle this?

Contributor

jnm2 commented Feb 11, 2017

@codito Thank you! I'm guessing this is a follow up from dotnet/sdk#833 (comment)?

If I understand correctly, the current guidance is to tell people not to target netstandard at all in the top-level test projects being executed, but rather multitarget them to specific platforms and versions like you're saying. This gives the test runner a DLL per platform and version and the test runner should be able to tell which platform and version to use by examining each DLL.

Here's a question I'm struggling with: if the NUnit framework projects take advantage of multitargeting, they'll probably consolidate to a single project targeting net20;net35;net40;net45;netstandard1.6;.NETPortable,Version=v4.5,Profile=Profile259.

What should the framework test projects should be targeting? Due to preprocessor conditions, each of these six framework targets runs slightly different code.

That means we'd want to:

  • run tests on the net20 framework DLL on net20
  • run tests on the net35 framework DLL on net35
  • run tests on the net40 framework DLL on net40
  • run tests on the net45 framework DLL on net45
  • run tests on the netstandard1.6 framework DLL on netcore1.0
  • run tests on the netstandard1.6 framework DLL on net47 (maybe)
  • run tests on the .NETPortable,Version=v4.5,Profile=Profile259 framework DLL on net45
  • run tests on the .NETPortable,Version=v4.5,Profile=Profile259 framework DLL on win8 or netcore1.0 (maybe)

In particular, the test projects need to end up running tests against net45 twice: once for the net45 framework DLL, and once for the portable framework DLL.

The simple transformation net20;net35;net40;net45;netstandard1.6;.NETPortable,Version=v4.5,Profile=Profile259 -> net20;net35;net40;net45;netcoreapp1.0;net45 doesn't really make sense anymore. What's a good way to handle this?

@codito

This comment has been minimized.

Show comment
Hide comment
@codito

codito Feb 11, 2017

.NETPortable,Version=v4.5,Profile=Profile259 is same as netstandard1.0 (see pcl vs netstandard spec).

Let's take the conflicting cases:

  1. run tests on the net45 framework DLL on net45
  2. run tests on the netstandard1.0 framework DLL on net45

Here dotnet tooling (nuget) will ensure 2 will never happen.

There's tool to find how nuget resolves the nearest dependency:
http://nugettoolsdev.azurewebsites.net/3.5.0-beta2-1484/get-nearest-framework?project=net45&package=netstandard1.0%0D%0Anet45

In the example above, NUnit will provide netstandard1.0 only to support test execution on win8. netstandard1.6 will be always picked up for netcoreapp1.0.

codito commented Feb 11, 2017

.NETPortable,Version=v4.5,Profile=Profile259 is same as netstandard1.0 (see pcl vs netstandard spec).

Let's take the conflicting cases:

  1. run tests on the net45 framework DLL on net45
  2. run tests on the netstandard1.0 framework DLL on net45

Here dotnet tooling (nuget) will ensure 2 will never happen.

There's tool to find how nuget resolves the nearest dependency:
http://nugettoolsdev.azurewebsites.net/3.5.0-beta2-1484/get-nearest-framework?project=net45&package=netstandard1.0%0D%0Anet45

In the example above, NUnit will provide netstandard1.0 only to support test execution on win8. netstandard1.6 will be always picked up for netcoreapp1.0.

@jnm2

This comment has been minimized.

Show comment
Hide comment
@jnm2

jnm2 Feb 11, 2017

Contributor

As far as the NUnit framework's own tests, they use project refs and not NuGet. I haven't looked but currently I don't think any tests are run on win8.
I'm pretty sure we will want a way to run the portable DLL tests on net45. I guess that will mean retaining a separate framework .csproj just for the portable build, and a separate test .csproj just to reference the separate portable framework .csproj. Is that the best option in the situation?

If someday target execution platforms could be specified per target, that would allow a single .csproj for the framework and a single .csproj for the test project which could target netstandard1.6 and netstandard1.0 and run each on a couple of platforms, as well as the net* targets each on its platform.

Contributor

jnm2 commented Feb 11, 2017

As far as the NUnit framework's own tests, they use project refs and not NuGet. I haven't looked but currently I don't think any tests are run on win8.
I'm pretty sure we will want a way to run the portable DLL tests on net45. I guess that will mean retaining a separate framework .csproj just for the portable build, and a separate test .csproj just to reference the separate portable framework .csproj. Is that the best option in the situation?

If someday target execution platforms could be specified per target, that would allow a single .csproj for the framework and a single .csproj for the test project which could target netstandard1.6 and netstandard1.0 and run each on a couple of platforms, as well as the net* targets each on its platform.

@codito

This comment has been minimized.

Show comment
Hide comment
@codito

codito Feb 12, 2017

I guess that will mean retaining a separate framework .csproj just for the portable build...

Yes, this seems like best option to me. The other option is manually copying over the netstandard1.0 framework dll along side net45 test, before running the tests.

Theoretically, netstandard1.0 should be tested on win8 only because that's the customer use case. I understand there is a practical aspect where win8 adds a new platform to test matrix and makes developer's life bit difficult, and net45 can be used there :)

codito commented Feb 12, 2017

I guess that will mean retaining a separate framework .csproj just for the portable build...

Yes, this seems like best option to me. The other option is manually copying over the netstandard1.0 framework dll along side net45 test, before running the tests.

Theoretically, netstandard1.0 should be tested on win8 only because that's the customer use case. I understand there is a practical aspect where win8 adds a new platform to test matrix and makes developer's life bit difficult, and net45 can be used there :)

enricosada added a commit to enricosada/FsUnit that referenced this issue Feb 17, 2017

use nunitlite for .netcore testing
nunit doesnt support yet `dotnet test` and netstandard lib (ref nunit/nunit3-vs-adapter#297 )

Is possible to use nunit lite, ref https://github.com/nunit/docs/wiki/NUnitLite-Runner
The test lib become a normal console app, and the nunit runner is embeeded (nunitlite.dll), and autostarted at main

enricosada added a commit to enricosada/FsUnit that referenced this issue Feb 17, 2017

use nunitlite for .netcore testing
nunit doesnt support yet `dotnet test` and netstandard lib (ref nunit/nunit3-vs-adapter#297 )

Is possible to use nunit lite, ref https://github.com/nunit/docs/wiki/NUnitLite-Runner
The test lib become a normal console app, and the nunit runner is embeeded (nunitlite.dll), and autostarted at main
@CharliePoole

This comment has been minimized.

Show comment
Hide comment
@CharliePoole

CharliePoole Apr 6, 2017

Member

What exactly does it mean to say "we can't support .net 3.5 when also .net standard/core is involved?" Clearly any given test assembly targets one or the other, not both.

Member

CharliePoole commented Apr 6, 2017

What exactly does it mean to say "we can't support .net 3.5 when also .net standard/core is involved?" Clearly any given test assembly targets one or the other, not both.

@OsirisTerje

This comment has been minimized.

Show comment
Hide comment
@OsirisTerje

OsirisTerje Apr 7, 2017

Member

Yes, but if a solution contains multiple projects, some using .net standard/core and others using framework 3.5 then it will require a testadapter that can cover both.

Member

OsirisTerje commented Apr 7, 2017

Yes, but if a solution contains multiple projects, some using .net standard/core and others using framework 3.5 then it will require a testadapter that can cover both.

@CharliePoole

This comment has been minimized.

Show comment
Hide comment
@CharliePoole

CharliePoole Apr 7, 2017

Member

Are we sure that's true? Why does the same adapter have to do both? Is the same process used for both?

Member

CharliePoole commented Apr 7, 2017

Are we sure that's true? Why does the same adapter have to do both? Is the same process used for both?

@OsirisTerje

This comment has been minimized.

Show comment
Hide comment
@OsirisTerje

OsirisTerje Apr 7, 2017

Member

We can of course run two different adapters for this. That is what I meant above, one for covering 3.5 targets, which would be the NUnit3 adapter series, and then the proposed NUnit4 adapter which would cover the .net standard/core projects, and anything but the 3.5 projects.
There is however an issue with tests that could possibly be covered by both adapters, and that is standard projects targeting .net 4.0 and above. There is no way the adapters can be "directed" to run only for a given project. They are all loaded in and executes over all the projects, running those they can run.
But do we really need to cover all these [fringe] cases?
If we skip support for .net 3.5 in a proposed 4 adapter, then people can still use the 3 adapter to run their stuff. And the likelihood of someone combining .net 3.5 and standard/core should be extremely small.

Member

OsirisTerje commented Apr 7, 2017

We can of course run two different adapters for this. That is what I meant above, one for covering 3.5 targets, which would be the NUnit3 adapter series, and then the proposed NUnit4 adapter which would cover the .net standard/core projects, and anything but the 3.5 projects.
There is however an issue with tests that could possibly be covered by both adapters, and that is standard projects targeting .net 4.0 and above. There is no way the adapters can be "directed" to run only for a given project. They are all loaded in and executes over all the projects, running those they can run.
But do we really need to cover all these [fringe] cases?
If we skip support for .net 3.5 in a proposed 4 adapter, then people can still use the 3 adapter to run their stuff. And the likelihood of someone combining .net 3.5 and standard/core should be extremely small.

@rprouse

This comment has been minimized.

Show comment
Hide comment
@rprouse

rprouse Apr 7, 2017

Member

The same adapter can support .NET 3.5 and .NET Core, it is just more work, so I wanted to make sure we needed 3.5.

  • Because 3.5 is older, I need more #if blocks in the code or more compatibility classes
  • Microsoft/msbuild#1333 means that I cannot build and pack the adapter using the dotnet command line, but instead need to use msbuild. This is mainly just redoing build.cake again and I have had trouble with MSBuild 14 getting pulled in instead of 15 on the CI machines. Nothing insurmountable, just more work 😄

I don't think maintaining two adapters going forward is a good idea unless we are saying the v3 adapter isn't going to see ongoing work. I think we should continue to move it forward as one project including our original plan of running the NUnit 2 tests.

I have a branch where I started the .NET 3.5 work. I will carry on with the intention of supporting 3.5.

Member

rprouse commented Apr 7, 2017

The same adapter can support .NET 3.5 and .NET Core, it is just more work, so I wanted to make sure we needed 3.5.

  • Because 3.5 is older, I need more #if blocks in the code or more compatibility classes
  • Microsoft/msbuild#1333 means that I cannot build and pack the adapter using the dotnet command line, but instead need to use msbuild. This is mainly just redoing build.cake again and I have had trouble with MSBuild 14 getting pulled in instead of 15 on the CI machines. Nothing insurmountable, just more work 😄

I don't think maintaining two adapters going forward is a good idea unless we are saying the v3 adapter isn't going to see ongoing work. I think we should continue to move it forward as one project including our original plan of running the NUnit 2 tests.

I have a branch where I started the .NET 3.5 work. I will carry on with the intention of supporting 3.5.

@OsirisTerje

This comment has been minimized.

Show comment
Hide comment
@OsirisTerje

OsirisTerje Apr 7, 2017

Member

Ok, this means we will have sort of the same code base, at least the files will be the same, but the #if blocks will separate things. I guess that is workable, as long as the blocks are not too big nor too many. But we will still have two releases then.

Member

OsirisTerje commented Apr 7, 2017

Ok, this means we will have sort of the same code base, at least the files will be the same, but the #if blocks will separate things. I guess that is workable, as long as the blocks are not too big nor too many. But we will still have two releases then.

@rprouse

This comment has been minimized.

Show comment
Hide comment
@rprouse

rprouse Apr 7, 2017

Member

@OsirisTerje yes, the code is very much the same in the adapter. I worked hard in the engine code to keep the API as similar as possible so that consumers of the API didn't have to care which platform they are targeting. The differences come down to differences in the underlying .NET API's like reflection or the lack of AppDomains in Core. Take a look at the PR if you are interested.

Member

rprouse commented Apr 7, 2017

@OsirisTerje yes, the code is very much the same in the adapter. I worked hard in the engine code to keep the API as similar as possible so that consumers of the API didn't have to care which platform they are targeting. The differences come down to differences in the underlying .NET API's like reflection or the lack of AppDomains in Core. Take a look at the PR if you are interested.

@rprouse

This comment has been minimized.

Show comment
Hide comment
@rprouse

rprouse Apr 8, 2017

Member

We can put the .NET 3.5 question to bed. I spent the night getting the build updated to support .NET 3.5. It is working and pushed to the PR.

Member

rprouse commented Apr 8, 2017

We can put the .NET 3.5 question to bed. I spent the night getting the build updated to support .NET 3.5. It is working and pushed to the PR.

@bernardbr

This comment has been minimized.

Show comment
Hide comment
@bernardbr

bernardbr Apr 8, 2017

Thanks @rprouse...

I'm working with a conversion of a project to NetCore and I stopped it to wait this.

Thank you very much!

Thanks @rprouse...

I'm working with a conversion of a project to NetCore and I stopped it to wait this.

Thank you very much!

@duncanawoods

This comment has been minimized.

Show comment
Hide comment
@duncanawoods

duncanawoods Apr 29, 2017

Whats the current status? Are .Net Core users having to migrate away from NUnit?

Whats the current status? Are .Net Core users having to migrate away from NUnit?

@rprouse

This comment has been minimized.

Show comment
Hide comment
@rprouse

rprouse Apr 30, 2017

Member

@duncanawoods we have working builds, see #313. As soon as that PR is fully reviewed and merged, I will create an alpha release to NuGet, hopefully this week.

Member

rprouse commented Apr 30, 2017

@duncanawoods we have working builds, see #313. As soon as that PR is fully reviewed and merged, I will create an alpha release to NuGet, hopefully this week.

@duncanawoods

This comment has been minimized.

Show comment
Hide comment
@duncanawoods

duncanawoods May 1, 2017

@rprouse great thanks - need a hand with anything?

@rprouse great thanks - need a hand with anything?

@rprouse

This comment has been minimized.

Show comment
Hide comment
@rprouse

rprouse May 1, 2017

Member

@duncanawoods once the alpha is released, I will need help testing it, so using in your projects and reporting issues 😄

Member

rprouse commented May 1, 2017

@duncanawoods once the alpha is released, I will need help testing it, so using in your projects and reporting issues 😄

@bernardbr

This comment has been minimized.

Show comment
Hide comment
@bernardbr

bernardbr May 1, 2017

@rprouse you can count on me!
Is this version will be at nuget.org?

@rprouse you can count on me!
Is this version will be at nuget.org?

@rprouse

This comment has been minimized.

Show comment
Hide comment
@rprouse

rprouse May 1, 2017

Member

@bernardbr yes, I will only be releasing the alphas to nuget.org. I will not release early alphas of the VSIX as VSIX does not support .NET Core (a limitation of the VSIX format, not NUnit).

Member

rprouse commented May 1, 2017

@bernardbr yes, I will only be releasing the alphas to nuget.org. I will not release early alphas of the VSIX as VSIX does not support .NET Core (a limitation of the VSIX format, not NUnit).

@bernardbr

This comment has been minimized.

Show comment
Hide comment
@bernardbr

bernardbr May 2, 2017

Thanks @rprouse!
I'll wait!

Thanks @rprouse!
I'll wait!

@rprouse

This comment has been minimized.

Show comment
Hide comment
@rprouse

rprouse May 3, 2017

Member

Update for those that are watching this issue, @OsirisTerje has done a fairly extensive code review. There are a few decisions that we need to make, but we are very close to being ready to do an alpha release of this.

Member

rprouse commented May 3, 2017

Update for those that are watching this issue, @OsirisTerje has done a fairly extensive code review. There are a few decisions that we need to make, but we are very close to being ready to do an alpha release of this.

@rprouse rprouse closed this in #313 May 3, 2017

@galvesribeiro

This comment has been minimized.

Show comment
Hide comment
@galvesribeiro

galvesribeiro May 4, 2017

@rprouse Great work! Please notify us here (or gitter) whenever you release something. There are much people depending on it. Thanks!

@rprouse Great work! Please notify us here (or gitter) whenever you release something. There are much people depending on it. Thanks!

@Shockolate

This comment has been minimized.

Show comment
Hide comment
@Shockolate

Shockolate May 4, 2017

@galvesribeiro Looks like 3.8.0-alpha1 has been pushed to nuget.org

@galvesribeiro Looks like 3.8.0-alpha1 has been pushed to nuget.org

@rprouse

This comment has been minimized.

Show comment
Hide comment
@rprouse

rprouse May 4, 2017

Member

The alpha has been pushed and the release notes should get most people started. https://github.com/nunit/nunit3-vs-adapter/releases/tag/3.8-alpha1

I plan on writing a more thorough explanation this afternoon or evening and will post a link when that is done.

Member

rprouse commented May 4, 2017

The alpha has been pushed and the release notes should get most people started. https://github.com/nunit/nunit3-vs-adapter/releases/tag/3.8-alpha1

I plan on writing a more thorough explanation this afternoon or evening and will post a link when that is done.

@bernardbr

This comment has been minimized.

Show comment
Hide comment
@bernardbr

bernardbr May 4, 2017

@rprouse
The debug functionality still doesn't work without the beta version of OmniSharp.
Need I do something different to make it work?

Thanks!

@rprouse
The debug functionality still doesn't work without the beta version of OmniSharp.
Need I do something different to make it work?

Thanks!

@rprouse

This comment has been minimized.

Show comment
Hide comment
@rprouse

rprouse May 4, 2017

Member

@bernardbr I haven't had a chance to figure out what is wrong with Visual Studio Code without the OmniSharp fixes. Hopefully they will ship it soon so I can forget about the fix on our end 😉

Member

rprouse commented May 4, 2017

@bernardbr I haven't had a chance to figure out what is wrong with Visual Studio Code without the OmniSharp fixes. Hopefully they will ship it soon so I can forget about the fix on our end 😉

@bernardbr

This comment has been minimized.

Show comment
Hide comment
@bernardbr

bernardbr May 4, 2017

Don't worry @rprouse ! Because with the last beta version of omnisharp the debug works fine! 😄
Thanks!

Don't worry @rprouse ! Because with the last beta version of omnisharp the debug works fine! 😄
Thanks!

@rprouse

This comment has been minimized.

Show comment
Hide comment
@rprouse

rprouse May 5, 2017

Member

For those following this issue who wants to start working with the new test adapter, I have written a blog post covering it's usage. Once everything settles down, I will migrate the core info into the docs, but for now, instructions are at http://www.alteridem.net/2017/05/04/test-net-core-nunit-vs2017/

Member

rprouse commented May 5, 2017

For those following this issue who wants to start working with the new test adapter, I have written a blog post covering it's usage. Once everything settles down, I will migrate the core info into the docs, but for now, instructions are at http://www.alteridem.net/2017/05/04/test-net-core-nunit-vs2017/

@yaakov-h

This comment has been minimized.

Show comment
Hide comment
@yaakov-h

yaakov-h May 5, 2017

Fairly straightforward, working well so far. Thanks @rprouse et al.

yaakov-h commented May 5, 2017

Fairly straightforward, working well so far. Thanks @rprouse et al.

@canton7

This comment has been minimized.

Show comment
Hide comment
@canton7

canton7 May 5, 2017

Once everything settles down, I will migrate the core info into the docs, but for now, instructions are at http://www.alteridem.net/2016/10/03/nunit-unit-tests/

@rprouse, did you mean http://www.alteridem.net/2017/05/04/test-net-core-nunit-vs2017/ ?

canton7 commented May 5, 2017

Once everything settles down, I will migrate the core info into the docs, but for now, instructions are at http://www.alteridem.net/2016/10/03/nunit-unit-tests/

@rprouse, did you mean http://www.alteridem.net/2017/05/04/test-net-core-nunit-vs2017/ ?

@Gerfaut

This comment has been minimized.

Show comment
Hide comment
@Gerfaut

Gerfaut May 5, 2017

Hi @rprouse, this new version is perfectly working when you run the tests both from dotnet test command or from Visual Studio! Thanks a lot for that!!

But it seems that debugging a selected test is not working (not sure debugging functionality was already foreseen to work in this alpha release).

nunit-380-alpha-test

Thanks a lot for all your heavy work on this last days!

Gerfaut commented May 5, 2017

Hi @rprouse, this new version is perfectly working when you run the tests both from dotnet test command or from Visual Studio! Thanks a lot for that!!

But it seems that debugging a selected test is not working (not sure debugging functionality was already foreseen to work in this alpha release).

nunit-380-alpha-test

Thanks a lot for all your heavy work on this last days!

@bunnu

This comment has been minimized.

Show comment
Hide comment
@bunnu

bunnu May 5, 2017

I already created an issue for the fatal error: #320

bunnu commented May 5, 2017

I already created an issue for the fatal error: #320

@daveabramson

This comment has been minimized.

Show comment
Hide comment
@daveabramson

daveabramson Jun 14, 2017

When I create NUnit tests in a .NET Core Library, 3.8.0 alpha1 is working for me in Visual Studio Test Explorer. But the tests do not get picked up in the Resharper Unit Test Explorer. When I create NUnit tests in a .NET Framework Library the tests are seen by both. XUnit tests are seen by both. My example code is attached.

image
image

ToyWithExamples.zip

When I create NUnit tests in a .NET Core Library, 3.8.0 alpha1 is working for me in Visual Studio Test Explorer. But the tests do not get picked up in the Resharper Unit Test Explorer. When I create NUnit tests in a .NET Framework Library the tests are seen by both. XUnit tests are seen by both. My example code is attached.

image
image

ToyWithExamples.zip

@CharliePoole

This comment has been minimized.

Show comment
Hide comment
@CharliePoole

CharliePoole Jun 14, 2017

Member

IIRC xUnit publish there own interfacing assembly for Resharper. We don't, but rely on Resharper to figure it out.

Member

CharliePoole commented Jun 14, 2017

IIRC xUnit publish there own interfacing assembly for Resharper. We don't, but rely on Resharper to figure it out.

@daveabramson

This comment has been minimized.

Show comment
Hide comment
@daveabramson

daveabramson Jun 14, 2017

Thanks @CharliePoole , I will go bug them :)

Thanks @CharliePoole , I will go bug them :)

@CharliePoole

This comment has been minimized.

Show comment
Hide comment
@CharliePoole

CharliePoole Jun 14, 2017

Member

The other option is for us to do as xUnit has done. I'm not sure we have enough people to add that to our list of projects, but maybe somebody will read this and volunteer. 😄

Member

CharliePoole commented Jun 14, 2017

The other option is for us to do as xUnit has done. I'm not sure we have enough people to add that to our list of projects, but maybe somebody will read this and volunteer. 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment