-
Notifications
You must be signed in to change notification settings - Fork 315
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
dotnet test swallows unit tests #2123
Comments
@livarcocc could you run this with diag logs enabled and share them here? |
@ShreyasRmsft I just transferred the issue here. @DerAlbertCom can you share diag logs here so that this can be further investigated? |
@ShreyasRmsft @livarcocc you now that there is a repository with an easy reproduction sample, and a lot more details https://github.com/DerAlbertCom/MisbehavingUnittestRunners But anyway :) here the logs. diag.host.19-08-12_19-59-49_08946_6.log And yes, the missing tests (Test2 and Test3) are also not in that logs. |
@DerAlbertCom i took a quick look. The host diag log that you see is basically open for the test adapters to use as they please. The test host process simply invokes the adapter and listens to exceptions if any. In this case the xunit adapter did not throw any exceptions. I know for a fact that mstest does log the reason/exception if any when discovery has issues. I would recommend you either try and debug through the xunit adapter to see why there are no logs. I've seen NUnit also printing messages like failed to discover tests due to failure in loading assembly etc. The mechanism for logging is present, in this case i think the xunit adapter is not leveraging it. |
@ShreyasRmsft thank for the information. I filed already the bugs under xunit/visualstudio.xunit#164 and xunit/visualstudio.xunit#165 no reaction so far. |
@DerAlbertCom since all of these are open source you should be able to just add a throw in the discovery code of the xunit adapter and see who is handling/catching it. |
From @DerAlbertCom on Friday, August 9, 2019 12:35:27 PM
dotnet swallows silently xunit tests (maybe also test from other test frameworks) when there is a problem on the test discovery. I filed the same under: xunit/visualstudio.xunit#164
Steps to reproduce
Checkout the repository https://github.com/DerAlbertCom/MisbehavingUnittestRunners run dotnet test. More details why the problem occurs is also there.
Expected behavior
At least the exception within the test discovery shows up. It would be better if it is possible to give more detail on exact type or test method.
Actual behavior
In the sample repository are 6 Unit Test, 2 of them get swallowed without any hint because of an exception within the test discoery on one test.
Environment data
Copied from original issue: dotnet/cli#12187
The text was updated successfully, but these errors were encountered: