Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow specifying Test Selection Language filters in VSTest runsettings #669

Merged
merged 1 commit into from Dec 13, 2019

Conversation

@mletterle
Copy link
Contributor

mletterle commented Oct 30, 2019

This allows using dotnet test as a drop in replacement for a
nunit-console run with the --where option.

i.e.:

nunit-console SomeTests.dll --where "cat == SomeCategory"
becomes
dotnet test SomeTest.csproj -- NUnit.TestsWhere="cat == SomeCategory"

Related: #425, #655

@mletterle mletterle force-pushed the mletterle:nunit-filter branch 2 times, most recently from 4701208 to 3358c8c Oct 30, 2019
@OsirisTerje

This comment has been minimized.

Copy link
Member

OsirisTerje commented Oct 30, 2019

@mletterle The idea of adding test filters to the runsettings is a good one, I like that one!
But, I don't really understand the connection with dotnet test, which already supports the --filter. Can you elaborate a bit on this?

Also, is this really two different things in one PR?

Do you see this PR as resolving issues with #425 and #655

@mletterle

This comment has been minimized.

Copy link
Contributor Author

mletterle commented Oct 30, 2019

@mletterle The idea of adding test filters to the runsettings is a good one, I like that one!
But, I don't really understand the connection with dotnet test, which already supports the --filter. Can you elaborate a bit on this?

Sure, the filter used by --filter is VSTest's filter dialect which is different then NUnit's native Test Selection Language

Not only is the syntax different, but as far as I can tell VSTest's filtering is far more limited than NUnit's native filtering.

This PR allows one to use NUnit's native Test Selection Language from the dotnet test command (and also via dotnet vstest)

Also, is this really two different things in one PR?

Do you see this PR as resolving issues with #425 and #655

The two issues have a common source, they're both about selecting which tests you actually want to run under NUnit.

#425 is related because this allows you to run dotnet test -- NUnit.TestsWhere="arbitrary Test Selection Language Filter"

#655 is related because, as far as I can tell it's actually not possible to use VSTest filter to select a specific parameterized test, but it is possible with the Test Selection Language, so for example you could do:

dotnet test -- NUnit.TestsWhere="test == 'My.Name.Space.TestFixture.TestMethod(5)'"

@OsirisTerje

This comment has been minimized.

Copy link
Member

OsirisTerje commented Nov 1, 2019

Thanks @mletterle ! Makes sense!

I'll add some review comments :-)

@OsirisTerje OsirisTerje self-requested a review Nov 1, 2019
src/NUnitTestAdapter/AdapterSettings.cs Outdated Show resolved Hide resolved
src/NUnitTestAdapter/NUnitTestFilterBuilder.cs Outdated Show resolved Hide resolved
src/NUnitTestAdapter/NUnitTestFilterBuilder.cs Outdated Show resolved Hide resolved
src/NUnitTestAdapterTests/TestExecutionTests.cs Outdated Show resolved Hide resolved
@mletterle mletterle force-pushed the mletterle:nunit-filter branch 2 times, most recently from c6e687a to 51acddb Nov 1, 2019
@mletterle mletterle requested a review from OsirisTerje Nov 8, 2019
@mletterle

This comment has been minimized.

Copy link
Contributor Author

mletterle commented Dec 13, 2019

@OsirisTerje Is there any further changes you'd like to see?

@OsirisTerje

This comment has been minimized.

Copy link
Member

OsirisTerje commented Dec 13, 2019

I think this is fine. I would like to have this out together with the other changes in 3.16. Have you merged the latest changes from master in here? And verified there are no crashes there ?

@OsirisTerje OsirisTerje added this to the 3.16 milestone Dec 13, 2019
This allows using `dotnet test` as a drop in replacement for a
nunit-console run with the `--where` option.

i.e.:

`nunit-console SomeTests.dll --where "cat == SomeCategory"`
becomes
`dotnet vstest SomeTests.dll -- NUnit.Where="cat == SomeCategory"`
or
`dotnet test SomeTests.csproj -- NUnit.Where="cat == SomeCategory"`
@mletterle mletterle force-pushed the mletterle:nunit-filter branch from 51acddb to caec049 Dec 13, 2019
@mletterle

This comment has been minimized.

Copy link
Contributor Author

mletterle commented Dec 13, 2019

I rebased the PR against current master, everything appears ok.

@OsirisTerje OsirisTerje merged commit 771790f into nunit:master Dec 13, 2019
4 checks passed
4 checks passed
NUnit 3 VSTest Adapter CI #20191213.1 succeeded
Details
WIP Ready for review
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
license/cla All CLA requirements met.
Details
@OsirisTerje

This comment has been minimized.

Copy link
Member

OsirisTerje commented Dec 13, 2019

Btw, are you writing blog posts? This change could deserve a post.

@mletterle mletterle deleted the mletterle:nunit-filter branch Dec 13, 2019
@mletterle

This comment has been minimized.

Copy link
Contributor Author

mletterle commented Dec 13, 2019

It's been a long time since I wrote a blog post ^_^

It's not a very complicated feature, I could try and put something together this weekend, if it would be helpful.

@OsirisTerje

This comment has been minimized.

Copy link
Member

OsirisTerje commented Dec 13, 2019

It's obvious to those close to the issue at hand :-) But not so for many others. I am often a bit surprised by how well received simple explanations can be.
So it would be awesome if you did! I'll refer to it in the release notes afterward too.

@mletterle

This comment has been minimized.

Copy link
Contributor Author

mletterle commented Dec 16, 2019

@OsirisTerje

This comment has been minimized.

Copy link
Member

OsirisTerje commented Dec 16, 2019

Awesome! Thanks a lot!

@OsirisTerje

This comment has been minimized.

Copy link
Member

OsirisTerje commented Jan 3, 2020

@mletterle Version 3.16 has been published. Links to your blogpost added in release notes and on the Trick and Trips page. :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.