Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When running dotnet test with filter, all tests are run and all console output is logged. #941

Closed
ptmclean opened this issue Jan 10, 2022 · 7 comments

Comments

@ptmclean
Copy link

ptmclean commented Jan 10, 2022

When running a test using the cli(dotnet test) and using a --filter all tests are run but only the results from the filtered tests are reported. A simple repro can be found here. The output from the test run only includes information from the matching test but the non-matching tests also run.

The NUnit version here is 3.13.2 and Adapter version 4.2.0 with dotnet 6.0.101 on an ARM Mac. I can't be sure it is not an engine issue, but changing the version of the adapter back to 4.1.0 returns to the previous behaviour.

There is another change with this version where the console output is logged for all tests whether they pass or fail. This may well be OS/console specific...
image

@JohnDoeKyrgyz
Copy link

Also seeing this issue on Windows

@OsirisTerje
Copy link
Member

Thanks @ptmclean and @JohnDoeKyrgyz .

This seems to happen when an assembly doesn't have any tests matching that filter. Please confirm if that is what you see.
I have made a fix for that issue, and have enclosed a dev version. Would be awesome if you could verify if that fixes it in your cases.

About the console output, that was a change introduced in Version 4.2.0, responding to issue #934.

NUnit3TestAdapter.4.2.1-dev.02-dbg.zip

@OsirisTerje OsirisTerje added this to the 4.2.1 milestone Jan 13, 2022
OsirisTerje added a commit that referenced this issue Jan 16, 2022
@Sebazzz
Copy link

Sebazzz commented Jan 18, 2022

This seems to happen when an assembly doesn't have any tests matching that filter. Please confirm if that is what you see.

Yes, and an assembly that actually doesn't contain any tests at all does output the warning as expected.

That NuPKG does fix the issue btw 👍

@ptmclean
Copy link
Author

Great thanks @OsirisTerje! Sorry I've been slow to get back to you. I think that works well. I've pushed up the reference to the test package in the repro repo.

Looks good to me!

@OsirisTerje
Copy link
Member

Thanks! I'll get it out as soon as possible :-)

peppy added a commit to peppy/osu that referenced this issue Jan 18, 2022
…solved

Tests have started to output too much log content, causing viewing CI
failures to be painfully impossible. Roll back for now.

Fix may be related to
nunit/nunit3-vs-adapter#941, although we don't
use filter.
@OsirisTerje OsirisTerje mentioned this issue Jan 18, 2022
OsirisTerje added a commit that referenced this issue Jan 19, 2022
* Fixing Issue #941

* updating acceptance tests

* Update tests
@OsirisTerje
Copy link
Member

Release 4.2.1 is on nuget.org.

@tmx7777
Copy link

tmx7777 commented Jan 24, 2022

Works for me as well. Great 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants