Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to test non-Core code on Linux #3073

Open
jskeet opened this issue May 17, 2016 · 31 comments

Comments

Projects
None yet
@jskeet
Copy link

commented May 17, 2016

(Apologies if this is the wrong repo for this; it's not clear to me whether this is xUnit-specific or the expected behaviour when testing on Linux.)

Background

I have projects which don't target core yet, but I still want to test on Travis. They work fine on RC1 (with dnx test and the old xUnit setup) and work fine with an updated project.json on Windows, but fail to run tests on Linux.

It's not clear whether the non-core runtimes/frameworks are expected to work on non-Windows platforms. Using dotnet run works fine for a console app with a TFM of net451, so it's clearly got some knowledge... but dotnet test fails.

Similar issues: #913, #1534.

Steps to reproduce

Create xUnit test project with a framework of net451, and run dotnet restore, dotnet build, dotnet test.

Sample project.json:

{
  "dependencies": {
    "xunit": "2.1.0",
    "dotnet-test-xunit": "1.0.0-rc2-build10015",
  },

  "testRunner": "xunit",

  "frameworks": {
    "net451": { }
  }
}

Sample C#:

using Xunit;

public class Test
{
    [Fact] public void Foo() {}
}

Expected behavior

Tests run (using Mono).

Actual behavior

Restore and build both work, but test fails:

dotnet-test Error: 0 : Microsoft.DotNet.Cli.Utils.CommandUnknownException: No executable found matching command "dotnet-test-xunit"  
  at Microsoft.DotNet.Cli.Utils.ProjectDependenciesCommandFactory.FindProjectDependencyCommands(String commandName, IEnumerable`1 commandArgs, String configuration, NuGetFramework framework, String outputPath, String buildBasePath, String projectDirectory)  
  at Microsoft.DotNet.Cli.Utils.ProjectDependenciesCommandFactory.Create(String commandName, IEnumerable`1 args, NuGetFramework framework, String configuration)  
  at Microsoft.DotNet.Tools.Test.ConsoleTestRunner.DoRunTests(ProjectContext projectContext, DotnetTestParams dotnetTestParams)  
  at Microsoft.DotNet.Tools.Test.TestCommand.DoRun(String[] args)

Environment data

dotnet --info output:

.NET Command Line Tools (1.0.0-preview1-002702)

Product Information:
 Version:     1.0.0-preview1-002702
 Commit Sha:  6cde21225e

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  15.10
 OS Platform: Linux
 RID:         ubuntu.15.10-x6
@onovotny

This comment has been minimized.

Copy link
Member

commented May 17, 2016

As far as I can tell, one issue is that there's no specific Mono RID, so there's no runtime exe that we can create in the dotnet-test-xunit package for this.

@onovotny

This comment has been minimized.

Copy link
Member

commented May 17, 2016

/cc @bradwilson @piotrpMSFT

@bradwilson

This comment has been minimized.

Copy link
Member

commented May 17, 2016

Smells like a tooling problem, not an xUnit.net problem.

@jskeet

This comment has been minimized.

Copy link
Author

commented May 17, 2016

So I reported it in the right repo by chance? Result :)

Do let me know if there's anything I can do to help diagnose it. I realize there are a lot of variables in play here.

@julielerman

This comment has been minimized.

Copy link

commented May 18, 2016

I'm getting this error ( No executable found matching command "dotnet-test-") as well but with dotnet core on osx. I'm not finding much via googlebing so adding onto this relatively similar thread ...because it puts me in such good company. :)

@onovotny

This comment has been minimized.

Copy link
Member

commented May 18, 2016

@julielerman can you share your project.json?

@julielerman

This comment has been minimized.

Copy link

commented May 18, 2016

I've been mucking with it but results are still the same:

{
    "version": "2.0.0-*",
    "description": "",
    "authors": [
        ""
    ],
    "dependencies": {
        "Microsoft.EntityFrameworkCore.InMemory": "1.0.0-rc2-final",
        "src": "2.0.0",
        "xunit": "2.1.0",
        "dotnet-test-xunit": "1.0.0-rc2-build10015"
    },
    "frameworks": {
        "netcoreapp1.0": {
            "dependencies": {
                "Microsoft.NETCore.App": {
                    "type": "platform",
                    "version": "1.0.0-rc2-3002702"
                }
            },
            "imports": [
                "dnxcore50",
                "portable-net45+win8"
            ]
        }
    }
}
@onovotny

This comment has been minimized.

Copy link
Member

commented May 18, 2016

So to be clear, the above works on Windows but not OSX?

@julielerman

This comment has been minimized.

Copy link

commented May 18, 2016

have not done any of this on windows. using vscode on a mac. Also, I have a full project that is running. I will create a 1+1=2 test and try to run that to be sure it's not a problem related to the dependency on that other project. Also, the only other ref I found to this error message was followed by "oh ..typo in project.json...sorry" but I can't see one and vscode is not reporting any issues. Thanks!

@ardalis

This comment has been minimized.

Copy link

commented May 18, 2016

dotnet-test should be dotnet test, if you're still seeing that error. If you're not typing that yourself, then it seems like you need to update your tools/SDK.

@bradwilson

This comment has been minimized.

Copy link
Member

commented May 18, 2016

You're missing the testRunner entry. This may help: http://xunit.github.io/docs/getting-started-dotnet-core.html

@julielerman

This comment has been minimized.

Copy link

commented May 18, 2016

@bradwilson gah! I saw that but was so focused on the dependencies section. Running up to my macbook to try it out and will let you know.

@julielerman

This comment has been minimized.

Copy link

commented May 18, 2016

got it. sorry for my little RTFM snafu, @bradwilson . Sorry @jskeet for piggy backing. You do have that specification in your project.json.

@alberto-riggi

This comment has been minimized.

Copy link

commented May 20, 2016

I have the exact same problem as @jskeet in windows with TFM net452. Maybe it's not related to linux?

project json:

{
  "version": "1.0.0-*",
  "description": "UnitTests Class Library",
  "authors": [ "albertori" ],

  "dependencies": {
    "xunit": "2.1.0",
    "dotnet-test-xunit": "1.0.0-rc2-build10015"
  },

  "frameworks": {
    "net452": {
      "dependencies": {
        "NSubstitute": "1.10.0"
      }
    }
  },
  "testRunner": "xunit"
}

dotnet --info output:

.NET Command Line Tools (1.0.0-preview1-002702)

Product Information:
 Version:     1.0.0-preview1-002702
 Commit Sha:  6cde21225e

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.10586
 OS Platform: Windows
 RID:         win10-x64
@jskeet

This comment has been minimized.

Copy link
Author

commented May 20, 2016

@alberto-riggi: Just to check - you're having this problem on Windows? It only bit me on Linux. (My tiny sample works on Windows with both net451 and net452.)

@alberto-riggi

This comment has been minimized.

Copy link

commented May 20, 2016

@jskeet , correct, on windows.
I saw there is another issue regarding this #1370 .
Your example in win works just targeting net452?
I've read somewhere that if you have net45x and aspnet core as TFMs it works.

@jskeet

This comment has been minimized.

Copy link
Author

commented May 20, 2016

@alberto-riggi Yes, it works with just net452 - see the code/config in my first post, but with net452. It might be worth you trying that too.

@alberto-riggi

This comment has been minimized.

Copy link

commented May 20, 2016

Thanks john, I had to use "dotnet-test-xunit": "1.0.0-rc3-build10019" though.
As pointed here
#1370 (comment)

@thohng

This comment has been minimized.

Copy link

commented May 22, 2016

Got it works with work around on Ubuntu 14.04
Project:

{
  "version": "1.0.0-*",

  "testRunner": "xunit",

  "dependencies": {
    "dotnet-test-xunit": "1.0.0-rc3-*",
    "Project": "*",
    "xunit": "2.1.0"
  },

  "frameworks": {
    "net451": {
      "dependencies": {
        "Microsoft.NETCore.Platforms": "1.0.1-rc2-*",
        "Moq": "4.2.1510.2205"
      }
    }
  }
}

Failed:
dotnet test test/ProjectFolderTest

Success:
mono test/ProjectFolderTest/bin/Debug/net451/ubuntu.14.04-x64/dotnet-test-xunit.exe test/ProjectFolderTest/bin/Debug/net451/ubuntu.14.04-x64/ProjectFolderTest.dll

@TheRealPiotrP TheRealPiotrP added this to the 1.0.0-rtm milestone May 26, 2016

@ghost

This comment has been minimized.

Copy link

commented May 26, 2016

I confirm that dotnet test does NOT work on linux when targetting net451.

Setup:

  • Debian 8 (x64)
  • Mono 4.2.3
  • Dotnet 1.0.0-preview1-002702

project.json:

{
  "version": "1.0.0-*",

  "testRunner": "xunit",

  "dependencies": {
    "xunit": "2.1.0",
    "dotnet-test-xunit": "1.0.0-rc3-build10019"
  },

  "frameworks": {
    "net451": { }
  }
}

Output of dotnet test:

sebok$ dotnet test
Project Atelier.Tests (.NETFramework,Version=v4.5.1) was previously compiled. Skipping compilation.
dotnet-test Error: 0 : Microsoft.DotNet.Cli.Utils.CommandUnknownException: No executable found matching command "dotnet-test-xunit"
  at Microsoft.DotNet.Cli.Utils.ProjectDependenciesCommandFactory.FindProjectDependencyCommands(String commandName, IEnumerable`1 commandArgs, String configuration, NuGetFramework framework, String outputPath, String buildBasePath, String projectDirectory)
  at Microsoft.DotNet.Cli.Utils.ProjectDependenciesCommandFactory.Create(String commandName, IEnumerable`1 args, NuGetFramework framework, String configuration)
  at Microsoft.DotNet.Tools.Test.ConsoleTestRunner.DoRunTests(ProjectContext projectContext, DotnetTestParams dotnetTestParams)
  at Microsoft.DotNet.Tools.Test.TestCommand.DoRun(String[] args)

However, the build created directory bin/Debug/net451/debian.8-x64, from which I can manually run dotnet-test-xunit.exe:

sebok$ cd bin/Debug/net451/debian.8-x64/
sebok$ mono dotnet-test-xunit.exe Atelier.Tests.dll
xUnit.net .NET CLI test runner (64-bit debian.8-x64)
  Discovering: Atelier.Tests
  Discovered:  Atelier.Tests
  Starting:    Atelier.Tests
  Finished:    Atelier.Tests
=== TEST EXECUTION SUMMARY ===
  Atelier.Tests  Total: 1, Errors: 0, Failed: 0, Skipped: 0, Time: 0,063s
@andrewlock

This comment has been minimized.

Copy link

commented May 29, 2016

Same behaviour as @sebok - dotnet test works on windows but no osx.10.11 using net451

  • osx.10.11
  • Mono 4.2.3
  • dotnet 1.0.0-preview1-002702

project.json:

{
    "version": "1.0.0-*",
    "dependencies": {
        "dotnet-test-xunit": "1.0.0-rc2-*",
        "xunit": "2.1.0"
    },
    "testRunner": "xunit",
    "frameworks": {
        "netcoreapp1.0": {
            "dependencies": {
                "Microsoft.NETCore.App": {
                    "version": "1.0.0-rc2-3002702",
                    "type": "platform"
                }
            },
            "imports": [
                "dnxcore50",
                "portable-net451+win8"
            ]
        },
        "net451": {}
    }
}

Running dotnet test fails with the following output:

dotnet-test Error: 0 : Microsoft.DotNet.Cli.Utils.CommandUnknownException: No executable found matching command "dotnet-test-xunit"
   at Microsoft.DotNet.Cli.Utils.ProjectDependenciesCommandFactory.FindProjectDependencyCommands(String commandName, IEnumerable`1 commandArgs, String configuration, NuGetFramework framework, String outputPath, String buildBasePath, String projectDirectory)
   at Microsoft.DotNet.Cli.Utils.ProjectDependenciesCommandFactory.Create(String commandName, IEnumerable`1 args, NuGetFramework framework, String configuration)
   at Microsoft.DotNet.Tools.Test.ConsoleTestRunner.DoRunTests(ProjectContext projectContext, DotnetTestParams dotnetTestParams)
   at Microsoft.DotNet.Tools.Test.TestCommand.DoRun(String[] args)

Running dotnet-test-xunit.exe from bin/Debug/net451/osx.10.11-x64 succeeds:

xUnit.net .NET CLI test runner (32-bit Desktop .NET osx.10.11-x86)
  Discovering: NetEscapades.AspNetCore.SecurityHeaders.Test
  Discovered:  NetEscapades.AspNetCore.SecurityHeaders.Test
  Starting:    NetEscapades.AspNetCore.SecurityHeaders.Test
  Finished:    NetEscapades.AspNetCore.SecurityHeaders.Test
=== TEST EXECUTION SUMMARY ===
   NetEscapades.AspNetCore.SecurityHeaders.Test  Total: 12, Errors: 0, Failed: 0, Skipped: 0, Time: 0.284s

@borgdylan

This comment has been minimized.

Copy link

commented Jul 3, 2016

If a net451 exe exists in the package, you will have to do a mono bootstrap script.

#!/usr/bin/env bash

SOURCE="${BASH_SOURCE[0]}"
while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink
  DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
  SOURCE="$(readlink "$SOURCE")"
  [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
done
DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"

exec mono "$DIR/dotnet-test-xunit.exe" "$@"
@borgdylan

This comment has been minimized.

Copy link

commented Jul 3, 2016

@onovotny @julielerman @andrewlock The script above may help you.

@borgdylan

This comment has been minimized.

Copy link

commented Jul 3, 2016

So just putting the script where the exe is will not help. I will have to fork the repo and add a makefile to install this on *nix machines.

@ac-esousa

This comment has been minimized.

Copy link

commented Jul 12, 2016

@borgdylan Thanks for your script. I'm using, with the following improvements:

  • Including your script at my test projects root directory

  • Putting the following configuration on projector.json:

    "buildOptions": {
        "copyToOutput": "dotnet-test-xunit"
    }
    
  • Adding the following line to .gitattributes:

    dotnet-test-xunit -text
    

This way, your script always gets copied to output directory, and I'm able to run test as I would normally would: dotnet test

@borgdylan

This comment has been minimized.

Copy link

commented Jul 12, 2016

This is a more elegant solution than the one I used.

@sunsided

This comment has been minimized.

Copy link

commented Sep 29, 2016

We were stumbling over the same problem recently. I'm referencing dotnet-text-xunit version 2.2.0-preview3-build1047 in a net461 project; on Ubuntu 14.04 LTS dotnet test fails with

No executable found matching command "dotnet-test-xunit"

However, when I copy ~/.nuget/packages/dotnet-test-xunit/.../runtimes/unix-x64/lib/net451/dotnet-test-xunit.exe to the output directory of the test project (e.g. runtimes/unix-x64/lib/net451/dotnet-test-xunit.exe) and then add @borgdylan's script to the same directory, dotnet test works.
That's an awful lot of work, however.

project.json looks like:

{
  "buildOptions": {
    "debugType": "portable",
    "preserveCompilationContext": true
  },

  "dependencies": {
    "dotnet-test-xunit": "2.2.0-preview3-build1047",
    "xunit": "2.2.0-beta3-build3401",
    "FluentAssertions": "4.14.0",
    "MyProject": { "target": "project" }
  },

  "testRunner": "xunit",

  "frameworks": {
    "net461": { }
  }
}
@RehanSaeed

This comment has been minimized.

Copy link

commented Oct 10, 2016

I has been some time since this issue was raised. Is the above nasty workaround the only real solution at this point? The alternative being to wait for a preview3 tooling release?

RehanSaeed added a commit to Dotnet-Boxed/Framework that referenced this issue Oct 13, 2016

RehanSaeed added a commit to RehanSaeed/Serilog.Exceptions that referenced this issue Oct 13, 2016

gregbty added a commit to gregbty/cqs-utils that referenced this issue Dec 15, 2016

Add build configurations for AppVeyor and Travis CI
Currently the test project will only run tests for .NET Core due to dotnet/cli#3073.

gregbty added a commit to gregbty/AspNetCore.RequestLogger that referenced this issue Dec 15, 2016

@blackdwarf blackdwarf modified the milestones: 2.0.0, 1.0.0-rtm Jan 20, 2017

@bravecobra

This comment has been minimized.

Copy link

commented Mar 19, 2017

The bash script workaround no longer works when upgrading to the latest .NET SDK 1.0.1. The tests do run when using mono xunit.console.exe test.dll, but dotnet test fails to find any "again", specifying "'TestAdapterPath' does not contain any test adapters", continues with "permission denied", then exits.

@livarcocc livarcocc modified the milestones: 2.0.0 M1, 2.0.0 May 9, 2017

@AdamPD

This comment has been minimized.

Copy link

commented Jun 29, 2017

@bravecobra - I have the exact same problem. Do you know of any workaround?

@TheAngryByrd

This comment has been minimized.

Copy link

commented Jul 23, 2017

@AdamPD if you can upgrade your xunit project up to 2.3.0-beta3 you can use the dotnet-xunit tool which I added support for mono https://www.nuget.org/packages/dotnet-xunit/ xunit/xunit#1213

@livarcocc livarcocc modified the milestones: 2.1.3xx, 2.1.4xx Mar 12, 2018

@livarcocc livarcocc modified the milestones: 2.1.4xx, 3.0.1xx Jun 28, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.