New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'dotnet test' vs. 'dotnet xunit' #1407

Closed
bgribaudo opened this Issue Aug 15, 2017 · 10 comments

Comments

Projects
None yet
8 participants
@bgribaudo

bgribaudo commented Aug 15, 2017

Hello,

What are the pros and cons of running xUnit tests using dotnet test vs. using dotnet xunit?

dotnet xunit offers more command-line options than dotnet test, but when those extra options aren't needed, is there a particular advantage to using one over the other or is the choice just personal preference? :-)

Thank you,
Ben

@rhyswat

This comment has been minimized.

Show comment
Hide comment
@rhyswat

rhyswat Aug 22, 2017

@bgribaudo a differentiator for me is that dotnet xunit can generate XML output for display in CI jobs.

rhyswat commented Aug 22, 2017

@bgribaudo a differentiator for me is that dotnet xunit can generate XML output for display in CI jobs.

@yangshengf

This comment has been minimized.

Show comment
Hide comment
@yangshengf

yangshengf Aug 31, 2017

The difference in the backend is rather large. With dotnet xunit it is a custom command meaning the handler gets all the commandline arguments, similar to xunit.console.exe. However, dotnet test passes through an msbuild wrapper similar to msbuild and visualstudio runner. It provides a common pattern regardless of test framework you use. Hence the extra commandline options are not available. The discussion is on dotnet/cli#3114

If you don't need the extra options it is up to you.

yangshengf commented Aug 31, 2017

The difference in the backend is rather large. With dotnet xunit it is a custom command meaning the handler gets all the commandline arguments, similar to xunit.console.exe. However, dotnet test passes through an msbuild wrapper similar to msbuild and visualstudio runner. It provides a common pattern regardless of test framework you use. Hence the extra commandline options are not available. The discussion is on dotnet/cli#3114

If you don't need the extra options it is up to you.

@stakx

This comment has been minimized.

Show comment
Hide comment
@stakx

stakx Oct 14, 2017

Contributor

One advantage of dotnet test over dotnet xunit is that the former can be run at the solution level, and it'll run unit tests for all test projects it finds. dotnet xunit, it seems, only works correctly if run from inside a specific test project directory.

(Perhaps I am missing something here, and dotnet xunit can also run tests for several projects. If so, please correct me.)

Contributor

stakx commented Oct 14, 2017

One advantage of dotnet test over dotnet xunit is that the former can be run at the solution level, and it'll run unit tests for all test projects it finds. dotnet xunit, it seems, only works correctly if run from inside a specific test project directory.

(Perhaps I am missing something here, and dotnet xunit can also run tests for several projects. If so, please correct me.)

@rhyswat

This comment has been minimized.

Show comment
Hide comment
@rhyswat

rhyswat Oct 19, 2017

@stakx that may be true of dotnet 2.0 and if so it’s a Good Thing.
It isn’t true of dotnet 1.1 unfortunately so you have to cd my-tests or tell it which test csproj file to use.

rhyswat commented Oct 19, 2017

@stakx that may be true of dotnet 2.0 and if so it’s a Good Thing.
It isn’t true of dotnet 1.1 unfortunately so you have to cd my-tests or tell it which test csproj file to use.

@stakx

This comment has been minimized.

Show comment
Hide comment
@stakx

stakx Oct 19, 2017

Contributor

@rhyswat:

you have to cd my-tests or tell it which test csproj file to use.

Unfortunately, this remains the same for dotnet 2.0.2 and xUnit 2.3.0. (I never figured out how to specify a project file it should use, only cd works, really.)

Contributor

stakx commented Oct 19, 2017

@rhyswat:

you have to cd my-tests or tell it which test csproj file to use.

Unfortunately, this remains the same for dotnet 2.0.2 and xUnit 2.3.0. (I never figured out how to specify a project file it should use, only cd works, really.)

@bradwilson

This comment has been minimized.

Show comment
Hide comment
@bradwilson

bradwilson Oct 19, 2017

Member

I never figured out how to specify a project file it should use, only cd works, really.

That's because what you want is not currently supported by the .NET SDK.

We (as a community) must wait for them (the .NET SDK team) to support global commands, instead of project-level commands.

Member

bradwilson commented Oct 19, 2017

I never figured out how to specify a project file it should use, only cd works, really.

That's because what you want is not currently supported by the .NET SDK.

We (as a community) must wait for them (the .NET SDK team) to support global commands, instead of project-level commands.

@vludax

This comment has been minimized.

Show comment
Hide comment
@vludax

vludax Nov 1, 2017

@stakx:

One advantage of dotnet test over dotnet xunit is that the former can be run at the solution level, and it'll run unit tests for all test projects it finds.

Although when dotnet test is run at the solution level, it attempts to run non-test projects, and complains that they are missing Microsoft.NET.Test.Sdk.

vludax commented Nov 1, 2017

@stakx:

One advantage of dotnet test over dotnet xunit is that the former can be run at the solution level, and it'll run unit tests for all test projects it finds.

Although when dotnet test is run at the solution level, it attempts to run non-test projects, and complains that they are missing Microsoft.NET.Test.Sdk.

@bradwilson bradwilson closed this Nov 11, 2017

@VictorioBerra

This comment has been minimized.

Show comment
Hide comment
@VictorioBerra

VictorioBerra Dec 26, 2017

I am currently struggling with this same thing. There are powershell/linux tricks that can automate finding projects and running tests but it is not as easy as it should be.

VictorioBerra commented Dec 26, 2017

I am currently struggling with this same thing. There are powershell/linux tricks that can automate finding projects and running tests but it is not as easy as it should be.

@FrederikNygaardSvendsen

This comment has been minimized.

Show comment
Hide comment
@FrederikNygaardSvendsen

FrederikNygaardSvendsen Feb 16, 2018

@vludax @VictorioBerra I had the same problem, i solved it by executing test by running a shell file like this:

#!/bin/bash
ls *Tests*/*.csproj | xargs -L1 dotnet test

This will run the dotnet test command for all .csproj files within a folder that matches on ./Tests/*.csproj. So this is run from the solution folder

FrederikNygaardSvendsen commented Feb 16, 2018

@vludax @VictorioBerra I had the same problem, i solved it by executing test by running a shell file like this:

#!/bin/bash
ls *Tests*/*.csproj | xargs -L1 dotnet test

This will run the dotnet test command for all .csproj files within a folder that matches on ./Tests/*.csproj. So this is run from the solution folder

@VictorioBerra

This comment has been minimized.

Show comment
Hide comment
@VictorioBerra

VictorioBerra Feb 16, 2018

I was able to solve my issue by making sure I was specifying netcoreapp2.0 in the xunit folder

VictorioBerra commented Feb 16, 2018

I was able to solve my issue by making sure I was specifying netcoreapp2.0 in the xunit folder

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment