New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
vstest console does not show console output from a test logger for "quiet" verbosity level #1580
Comments
@smadala, @pvlakshm could you please take a look? It looks like a breaking change on which our customers complains: https://youtrack.jetbrains.com/issue/TW-54720 |
@dtretyakov @NikolayPianikov Thanks for reporting the issue. 2.2.100-preview1
2.1.200-preview-007474
To fix the issue, We will make verbosity value minimal by default same as dotnet cli 2.1.
@dtretyakov I'm not sure making verbosity value to minimal fixes this. |
@smadala Test Logger console output should be visible for ANY verbosity level
See the attached sample |
@NikolayPianikov Thanks for repro, We will look into it. @mayankbansal018 Already working on default verbosity level to minimal. |
I’ve also found this problem, which lead to this issue in the CLI: dotnet/cli#9229 |
@smadala The issue of the CLI passing the wrong default verbosity ( |
@peterhuene , we also made change to set default verbosity to minimal instead of quite, in PR dotnet/cli#9211, I hope the changes we made would still be relevant? |
That change affects the MSBuild verbosity as well, which is likely undesirable. The fix we took for RTM restores the previous behavior of being quiet for MSBuild and "unspecified" for VSTest (i.e. whatever default VSTest uses). Users that specify a |
I'll revert my fix then, thanks for the update @peterhuene , will close this issue as well. |
@mayankbansal018 Does quiet mode show stdOut messages from loggers? |
@NikolayPianikov quite mode does not show any messages from logger, quiet mode only shows passed, & failed count. |
I think it is a breaking change. Some CI are using stdOut for integration and obviously |
@NikolayPianikov we never used to show messages printed via Console.WriteLine(stdout) in CLI, it was always in detailed mode, the above change restores the behavior back to "minimal" as default, where we only show stack trace for failed messages + passed failed count |
As a user of dotnet test directly, from an interactive perspective I want stdout and stacks for failed tests only to keep the terminal uncluttered, but in CI want all tests output so tools like TeamCity can display the total passed/fail count on a per-test basis, rather than in aggregate, as well as stdout and stacks for failed tests. |
@martincostello for users who need complete data, they can simply pass -verbosity:detailed as input to dotnet cli. |
That wasn’t required for 2.1.1xx and earlier versions, and also that would increase the MSBuild verbosity, which I don’t want to do. As mentioned above, this is a breaking change for CI integrations that parse the output, like TeamCity. |
@mayankbansal018 vstest shows stdOut from loggers in all modes (including |
@mayankbansal018 may be there is some API to make possible to write down to stdOut without any filtrations? |
@NikolayPianikov, so far when I mean logger I was only referring to Console logger, I hope you also meant the same. Further I tried out SDK 2.1.104, & below are my observations:
Quiet:
Normal:
|
@mayankbansal018 it is not so important for us which information about tests is appeared in the stdOut because of we are skipping it from all loggers excepted our. Can I rely on this? |
@NikolayPianikov , as per @peterhuene comment , the behavior is restored as before where msbuild verbosity is set to quite, & vstest to unspecified(default:minimal). Which means that messages your were receiving in logger earlier should be restored. |
@mayankbansal018 thank you |
@mayankbansal018 another one issue: |
@NikolayPianikov , please create a new issue, & if possible also attach a sample repro. |
Description
The test logger produces a console output
vstest console does not show this output for default verbosity level (quiet)
Steps to reproduce
Install .NET Core sdk-2.1.300-rc1 or 2.1.200
Run tests from sample
Messages like
##teamcity[testStarted
should be in the stdOutExpected behavior
Test Logger console output should be visible for any verbosity level
Actual behavior
Test Logger console output is not visible for
quiet
verbosity levelDiagnostic logs
quiet.txt
normal.txt
Environment
Windows 10
dotnet --version
2.1.300-rc1-008673
2.1.200
The text was updated successfully, but these errors were encountered: