Test Adapter for Google Test cannot execute tests

[spebl]

Description

Google Tests cannot be run using the Test Adapter for Google Test that ships with Visual Studio with C++ installed.

Steps to reproduce

Using a Visual studio 16.10 preview build with "Desktop development with C++ installed".
File->New->Project and create a Google Test project choosing all the default options.
Open the Test Explorer.
Build the sample test.
Notice the test is discovered.
Run the test in the Test Explorer.

Expected behavior

Test is successfully run.

Actual behavior

The test is not run, and there is an error in the Tests output window pane "Could not find test executor with URI 'executor://testadapterforgoogletest/v1'. Make sure that the test executor is installed and supports .net runtime version ."

Diagnostic logs

I’ve enabled “Trace” logs from Test->Options->Logging Level. That puts out diagnostic test platform logs under %temp%\TestPlatformLogs. Looking at the vstest.console log (logs.txt) I see that the Test Explorer is initializing the platform with google test adapter:

TpTrace Information: 0 : 16128, 1, 2021/04/08, 06:15:13.075, 10351613562, vstest.console.exe, DesignModeClient.ProcessRequests: Processing Message: (Extensions.Initialize) -> [
"C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2019\INTPREV\COMMON7\IDE\COMMONEXTENSIONS\MICROSOFT\EBF\TESTEXPLORER\Microsoft.VisualStudio.Workspace.ExternalBuildFramework.TestProvider.dll",
"C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2019\INTPREV\COMMON7\IDE\EXTENSIONS\FHJUPEZ5.Z5O\BoostTestAdapter.dll",
"C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2019\INTPREV\COMMON7\IDE\EXTENSIONS\FYY32XX5.ELU\GoogleTestAdapter.TestAdapter.dll",
"C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2019\INTPREV\COMMON7\IDE\EXTENSIONS\MICROSOFT\NODEJSTOOLS\TESTADAPTER\Microsoft.NodejsTools.TestAdapter.dll",
"C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2019\INTPREV\COMMON7\IDE\EXTENSIONS\MICROSOFT\PYTHON\CORE\Microsoft.PythonTools.TestAdapter.Executor.dll"
]

The discovery process logs also indicate that the adapter is passed on to it (logs.host*_6.txt):

TpTrace Information: 0 : 13920, 7, 2021/04/08, 06:15:13.712, 10357985541, testhost.x86.exe, TestRequestHandler.ProcessRequests: received message: (TestDiscovery.Initialize) -> [
"C:\Program Files (x86)\Microsoft Visual Studio\2019\intprev\Common7\IDE\Extensions\TestPlatform\Extensions\Microsoft.Cci.dll",
….
"C:\Program Files (x86)\Microsoft Visual Studio\2019\intprev\Common7\IDE\Extensions\TestPlatform\Extensions\Microsoft.VisualStudio.TraceDataCollector.dll",
"C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2019\INTPREV\COMMON7\IDE\COMMONEXTENSIONS\MICROSOFT\EBF\TESTEXPLORER\Microsoft.VisualStudio.Workspace.ExternalBuildFramework.TestProvider.dll",
"C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2019\INTPREV\COMMON7\IDE\EXTENSIONS\FHJUPEZ5.Z5O\BoostTestAdapter.dll",
"C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2019\INTPREV\COMMON7\IDE\EXTENSIONS\FYY32XX5.ELU\GoogleTestAdapter.TestAdapter.dll",
"C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2019\INTPREV\COMMON7\IDE\EXTENSIONS\MICROSOFT\NODEJSTOOLS\TESTADAPTER\Microsoft.NodejsTools.TestAdapter.dll",
"C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2019\INTPREV\COMMON7\IDE\EXTENSIONS\MICROSOFT\PYTHON\CORE\Microsoft.PythonTools.TestAdapter.Executor.dll"
]

The test execution process logs on the other hand do not get initialized with the adapter(logs.host*_10.txt):

TpTrace Information: 0 : 3600, 8, 2021/04/08, 06:15:16.140, 10382260681, testhost.x86.exe, TestRequestHandler.ProcessRequests: received message: (TestExecution.Initialize) -> [
"C:\Program Files (x86)\Microsoft Visual Studio\2019\intprev\Common7\IDE\Extensions\TestPlatform\Extensions\Microsoft.Cci.dll",
…
"C:\Program Files (x86)\Microsoft Visual Studio\2019\intprev\Common7\IDE\Extensions\TestPlatform\Extensions\Microsoft.VisualStudio.TraceDataCollector.dll",
"C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2019\INTPREV\COMMON7\IDE\COMMONEXTENSIONS\MICROSOFT\EBF\TESTEXPLORER\Microsoft.VisualStudio.Workspace.ExternalBuildFramework.TestProvider.dll",
"C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2019\INTPREV\COMMON7\IDE\EXTENSIONS\MICROSOFT\NODEJSTOOLS\TESTADAPTER\Microsoft.NodejsTools.TestAdapter.dll"
]

Environment

Visual studio 16.10 Preview 3.0

AB#1315248

[Sanan07]

I was able to repro the issue, strange thing is that it works well in VS 16.10.0 Preview 1.0.
Maybe Medeni Baykal (@Haplois) changes related to fqn might cause it, cause I do not recall any change which we did recently could result this issue.

[Haplois]

Sanan Yuzbashiyev (@Sanan07), we fixed a couple of issues with FQN on our v16.10 preview bits. Could you test with that again?

[Sanan07]

Medeni Baykal (@Haplois) I tested in 16.10.0 Preview 3.0 [31223.327.main] still does not work, while in 16.10.0 Preview 2.1 works well.

[Haplois]

Sanan Yuzbashiyev (@Sanan07), can you check the exact TP version on that version?

[Sanan07]

VS version which does not work shows TP version 16.10.0, VS which works has 16.10.0-dev.

[jakubch1]

I was able to reproduce it and it looks like TestWindow issue:

Before discovery TP receives:
TpTrace Information: 0 : 37296, 1, 2021/04/27, 08:44:41.064, 6933550624253, vstest.console.exe, DesignModeClient.ProcessRequests: Processing Message: (Extensions.Initialize) -> [
"C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2019\PREVIEW\COMMON7\IDE\COMMONEXTENSIONS\MICROSOFT\EBF\TESTEXPLORER\Microsoft.VisualStudio.Workspace.ExternalBuildFramework.TestProvider.dll",
"C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2019\PREVIEW\COMMON7\IDE\EXTENSIONS\FHJUPEZ5.Z5O\BoostTestAdapter.dll",
"C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2019\PREVIEW\COMMON7\IDE\EXTENSIONS\RRFTGT4X.BTP\GoogleTestAdapter.TestAdapter.dll"
]

but before test execution TW sends additional request:

TpTrace Information: 0 : 37296, 1, 2021/04/27, 08:44:57.398, 6933713967761, vstest.console.exe, DesignModeClient.ProcessRequests: Processing Message: (Extensions.Initialize) -> [
"C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2019\PREVIEW\COMMON7\IDE\COMMONEXTENSIONS\MICROSOFT\EBF\TESTEXPLORER\Microsoft.VisualStudio.Workspace.ExternalBuildFramework.TestProvider.dll"
]

Second request doesn't contain Google Test Adapter and this is the reason why we do not execute tests.

In 16.9 there TW is not sending any additional Extensions.Initialize before test execution.

What is interesting if you keep changing x86 -> x64 -> x86 before each test execution then it works correctly in 16.10, but if you run it second time keeping same architecture it fails.

[spebl]

This also seems to be happening with the Test Adapter for Boost.Test, so it looks to be bigger than just a Google Test issue.

[Sanan07]

After TW fix, tested for Google Test Adapter in VS 16.10 Preview 3.0 [31230.346.main] and works fine. Spencer Bloom (@spebl) please check it.

[spebl]

Yep! Looks to be working now in the latest build. Thank you!