vstest console does not show console output from a test logger for "quiet" verbosity level

[NikolayPianikov]

Description

The test logger produces a console output
vstest console does not show this output for default verbosity level (quiet)

Steps to reproduce

Install .NET Core sdk-2.1.300-rc1 or 2.1.200
Run tests from sample
Messages like ##teamcity[testStarted should be in the stdOut

Expected behavior

Test Logger console output should be visible for any verbosity level

Actual behavior

Test Logger console output is not visible for quiet verbosity level

Diagnostic logs

quiet.txt
normal.txt

Environment

Windows 10
dotnet --version
2.1.300-rc1-008673
2.1.200

[dtretyakov]

@smadala, @pvlakshm could you please take a look? It looks like a breaking change on which our customers complains: https://youtrack.jetbrains.com/issue/TW-54720

[smadala]

@dtretyakov @NikolayPianikov Thanks for reporting the issue.
Looks like there is behavior difference between how the dotnet cli executing VSTestTask(Additional property -property:VSTestVerbosity=quiet).

2.2.100-preview1

"dotnet.exe" exec C:\Users\samadala\src\vstest\tools\dotnet\sdk\2.2.100-preview1-008636\MSBuild.dll -maxcpucount -verbosity:m -restore -target:VSTest -verbosity:quiet -nodereuse:false -nologo C:\Users\samadala\src\tmp\core-cc\core-cc.csproj -property:VSTestVerbosity=quiet -distributedlogger:Microsoft.DotNet.Tools.MSBuild.MSBuildLogger,C:\Users\samadala\src\vstest\tools\dotnet\sdk\2.2.100-preview1-008636\dotnet.dll*Microsoft.DotNet.Tools.MSBuild.MSBuildForwardingLogger,C:\Users\samadala\src\vstest\tools\dotnet\sdk\2.2.100-preview1-008636\dotnet.dll

2.1.200-preview-007474

"dotnet.exe" exec "C:\Program Files\dotnet\sdk\2.1.200-preview-007474\MSBuild.dll" /m /v:m /restore /t:VSTest /v:quiet /nologo C:\Users\samadala\src\tmp\core-cc\core-cc.csproj "/Logger:Microsoft.DotNet.Tools.MSBuild.MSBuildLogger,C:\Program Files\dotnet\sdk\2.1.200-preview-007474\dotnet.dll"

To fix the issue, We will make verbosity value minimal by default same as dotnet cli 2.1.

Now the results at least report the build as 'Failed' but I still don't get any test results.

@dtretyakov I'm not sure making verbosity value to minimal fixes this.

[NikolayPianikov]

@smadala Test Logger console output should be visible for ANY verbosity level
It is a breaking change:

• it is working as expected for SDK 2.1.104
• it is not working for SDK 2.1.200

See the attached sample

[smadala]

@NikolayPianikov Thanks for repro, We will look into it. @mayankbansal018 Already working on default verbosity level to minimal.

[martincostello]

I’ve also found this problem, which lead to this issue in the CLI: dotnet/cli#9229

[peterhuene]

@smadala The issue of the CLI passing the wrong default verbosity (VSTestVerbosity=quiet instead of not defining the property) to VSTest has been corrected for 2.1 RTM.

[mayankbansal018]

@peterhuene , we also made change to set default verbosity to minimal instead of quite, in PR dotnet/cli#9211, I hope the changes we made would still be relevant?

[peterhuene]

That change affects the MSBuild verbosity as well, which is likely undesirable. The fix we took for RTM restores the previous behavior of being quiet for MSBuild and "unspecified" for VSTest (i.e. whatever default VSTest uses). Users that specify a --verbosity option on the command line will have it still passed through to VSTest through the VSTestVerbosity property.

[mayankbansal018]

I'll revert my fix then, thanks for the update @peterhuene , will close this issue as well.

[NikolayPianikov]

@mayankbansal018 Does quiet mode show stdOut messages from loggers?