[Quesiton] How to run 'dotnet test' without cli?

[livarcocc]

From @chuckries on October 24, 2017 23:35

I am working on porting the debugger used by the VS Code C# Extension to linux-arm. Today we depend on the 'dotnet test' command to run our unit tests. This will not work for running our unit tests on linux-arm because there is currently no cli available.

I thought I would be able to build the test project on windows and run it on linux-arm against a private shared framework. This works for console apps, but I can't get it to work for a test app.

How can I get a test runner in the output of my test project build that I can run on arm against a shared framework? I thought doing a stand-alone publish would include the necessary pieces, but it doesn't have a test runner in the outputs. I took a quick look at 'dotnet-vstest' form the cli, and it looks like the runner is included in the SDK installation, which I do not have on linux-arm. Is there any way to publish 'vstest.console.dll' with the app? Is there a PackageReference I can add to include the test runner in the build/publish output?

Copied from original issue: dotnet/cli#7898

[livarcocc]

I do not believe that's possible today. The test project is not self reliant. In the sense that it does not contain the runner that knows how to run the tests.

Still, I will move this issue to microsoft/vstest, maybe they will have some idea there.

[codito]

@chuckries please use Microsoft.TestPlatform.Portable nuget package from https://dotnet.myget.org/F/vstest/api/v3/index.json myget feed. This tools nuget package provides tools\netcoreapp2.0\vstest.console.dll test runner. It can be directly used with a 2.0 shared runtime.

[chuckries]

I imagine this is a step in the right direction, but it's still not accomplishing what I'm trying to do. I want to build/publish a project on a machine with an SDK, and then run the outputs on a machine that only has a Shared Framework. When I include Microsoft.TestPlatform.Portable as a DotNetCliToolReference, I do not see vstest.console.dll in the build output or the publish output. Here is what I'm trying:

C:\Users\chuck\Source\scratch>dotnet --info
.NET Command Line Tools (2.0.2)

Product Information:
 Version:            2.0.2
 Commit SHA-1 hash:  a04b4bf512

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.16299
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\2.0.2\

Microsoft .NET Core Shared Framework Host

  Version  : 2.0.0
  Build    : e8b8861ac7faf042c87a5c2f9f2d04c98b69f28d

1. dotnet new xunit
2. Add the following to the .csproj:

  <ItemGroup>
    <DotNetCliToolReference Include="Microsoft.TestPlatform.Portable" Version="15.6.0-preview-20171024-04"/>
  </ItemGroup>

3. Add the following NuGet.config:

<configuration>
  <packageSources>
    <add key="vstest-myget" value="https://dotnet.myget.org/F/vstest/api/v3/index.json"/>
  </packageSources>
</configuration>

4. dotnet publish -r linux-arm

At this point I have a bin\Debug\netcoreapp2.0\linxu-arm\publish dir with all the dependencies needed to run on linux-arm, but the vstest.console.dll is not included. Without the test runner in the build/publish output, I do not know how I can build this test project and run the tests on a machine that only has a shared framework and no SDK.

Here is the listing of the publish output:

C:\Users\chuck\Source\scratch\bin\Debug\netcoreapp2.0\linux-arm\publish>dir
 Volume in drive C is Local Disk
 Volume Serial Number is D079-03A9

 Directory of C:\Users\chuck\Source\scratch\bin\Debug\netcoreapp2.0\linux-arm\publish

10/25/2017  08:46 AM    <DIR>          .
10/25/2017  08:46 AM    <DIR>          ..
10/25/2017  08:46 AM    <DIR>          cs
10/25/2017  08:46 AM    <DIR>          de
10/25/2017  08:46 AM    <DIR>          es
10/25/2017  08:46 AM    <DIR>          fr
10/25/2017  08:46 AM    <DIR>          it
10/25/2017  08:46 AM    <DIR>          ja
10/25/2017  08:46 AM    <DIR>          ko
02/06/2017  12:16 PM            21,488 Microsoft.DotNet.PlatformAbstractions.dll
02/06/2017  12:16 PM            58,352 Microsoft.Extensions.DependencyModel.dll
06/28/2017  04:51 AM            68,608 Microsoft.TestPlatform.CommunicationUtilities.dll
06/28/2017  04:51 AM            32,256 Microsoft.TestPlatform.CoreUtilities.dll
06/28/2017  04:51 AM           105,984 Microsoft.TestPlatform.CrossPlatEngine.dll
06/28/2017  04:51 AM            15,872 Microsoft.TestPlatform.PlatformAbstractions.dll
06/28/2017  04:51 AM           114,688 Microsoft.VisualStudio.TestPlatform.Common.dll
06/28/2017  04:51 AM           114,176 Microsoft.VisualStudio.TestPlatform.ObjectModel.dll
02/06/2017  12:16 PM           468,480 Newtonsoft.Json.dll
10/25/2017  08:46 AM    <DIR>          pl
10/25/2017  08:46 AM    <DIR>          pt-BR
10/25/2017  08:46 AM    <DIR>          ru
10/25/2017  08:46 AM            82,660 scratch.deps.json
10/25/2017  08:46 AM             4,096 scratch.dll
10/25/2017  08:46 AM               564 scratch.pdb
10/25/2017  08:46 AM               154 scratch.runtimeconfig.json
06/11/2016  11:15 PM           118,520 System.Xml.XPath.XmlDocument.dll
06/28/2017  04:51 AM            10,752 testhost.dll
10/25/2017  08:46 AM    <DIR>          tr
08/09/2016  08:37 AM            13,824 xunit.abstractions.dll
02/19/2017  04:18 PM            53,248 xunit.assert.dll
02/19/2017  04:18 PM            37,888 xunit.core.dll
02/19/2017  04:18 PM           189,952 xunit.execution.dotnet.dll
02/19/2017  04:24 PM            33,792 xunit.runner.reporters.netstandard15.dll
02/19/2017  04:24 PM           169,472 xunit.runner.utility.netstandard15.dll
02/19/2017  04:24 PM            53,760 xunit.runner.visualstudio.dotnetcore.testadapter.dll
10/25/2017  08:46 AM    <DIR>          zh-Hans
10/25/2017  08:46 AM    <DIR>          zh-Hant
              22 File(s)      1,768,586 bytes
              15 Dir(s)  262,814,564,352 bytes free

[chuckries]

@codito I think I have figured it out for the time being, but it is not as streamlined as I had hoped. I had to obtain the Microsoft.TestPlatform.Portable nuget package on Windows and copy the contents of the tools\netcoreapp2.0\ onto linux-arm. Then I was able to run dotnet vstest.console.dll on the shared framework against my test project. I was hoping that dotnet publish would do the heavy lifting for me, but apparently it won't. Thanks for the help.

[codito]

@chuckries I think the runner assemblies are not published with dotnet publish because it is a regular tools nuget package. I have created a separate issue for the dotnet-cli tools support: #1228.

[ghost]

@chuckries I followed your indications, in a .NET Core 2 solution published on a Raspberry PI, and I could get to run this kind of command:
dotnet xxx/vstest.console.dll xxx/libraryWithTests.dll
But I obtain the following result:
No suitable test runtime provider found for this run.
Could you please just have any ideas on this?
Thanks in advance...

[chuckries]

@andreabioli This has actually gotten a lot easier now that there is a full .NET Core SDK (as opposed to just the runtime) for linux-arm. The .NET Core 2.1 RC1 SDK for linux-arm can be downloaded here: https://dotnetcli.blob.core.windows.net/dotnet/Sdk/2.1.300-rc1-008673/dotnet-sdk-2.1.300-rc1-008673-linux-arm.tar.gz

With this SDK you can build and test your project directly on the raspberry pi rather than having to publish it for linux-arm on another platform. Use dotnet test to build and run the test project on arm. If you have already built the test binary for arm and just want to run it, you can use dotnet vstest <path/to/test.dll>

[ghost]

Oh, wonderful news! :-) Thank you very much, I will give a look!

[csMACnz]

I played with this for a while because I really wanted to avoid the sdk for no particular reason.

I figured out I could use https://www.nuget.org/packages/xunit.console/ and call it's main from my main, hardcoding my app as the test dll. Publish your app and it becomes self executing without the test framework.

https://stackoverflow.com/a/64888158/2118268

Not sure if the lack of future development on xunit.console will become an issue at some point, but seems to work for me for netcoreapp3.1 and net5 xunit test apps.