Failure to create CodeQL database with latest Visual Studio (17.8.1)

[matlimatli]

Description of the issue
After upgrading the Visual Studio environment from 17.4 to 17.8.1, we can not create a CodeQL database.
It looks like the compiler detection is failing. Se extract of build below.
Full logging available as attachment:
codeql.zip

[2023-11-27 22:01:46] [build-stdout]   C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.38.33130\bin\HostX64\x64\CL.exe /c /Zi /nologo /W3 /WX- /diagnostics:column /sdl /O2 /Oi /GL /D NDEBUG /D _CONSOLE /D _UNICODE /D UNICODE /Gm- /EHsc /MD /GS /Gy /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /permissive- /Fo"x64\Release\\" /Fd"x64\Release\vc143.pdb" /external:W3 /Gd /TP /FC /errorReport:queue HelloWorld.cpp
[2023-11-27 22:01:46] [build-stdout]   HelloWorld.cpp
[2023-11-27 22:01:46] [build-stdout]   Microsoft (R) C/C++ Optimizing Compiler Version 19.38.33130 for x64
[2023-11-27 22:01:46] [build-stdout]   Copyright (C) Microsoft Corporation.  All rights reserved.
[2023-11-27 22:01:46] [build-stdout] cl : command line  warning D9002: ignoring unknown option '--version' [C:\helloworld\HelloWorld\HelloWorld.vcxproj]
[2023-11-27 22:01:46] [build-stdout] cl : command line  error D8003: missing source filename [C:\helloworld\HelloWorld\HelloWorld.vcxproj]
[2023-11-27 22:01:46] [build-stdout]   Microsoft (R) C/C++ Optimizing Compiler Version 19.38.33130 for x64
[2023-11-27 22:01:46] [build-stdout]   Copyright (C) Microsoft Corporation.  All rights reserved.
[2023-11-27 22:01:46] [build-stdout] cl : command line  warning D9002: ignoring unknown option '--help' [C:\helloworld\HelloWorld\HelloWorld.vcxproj]
[2023-11-27 22:01:46] [build-stdout] cl : command line  error D8003: missing source filename [C:\helloworld\HelloWorld\HelloWorld.vcxproj]
[2023-11-27 22:01:46] [build-stdout] Done Building Project "C:\helloworld\HelloWorld\HelloWorld.vcxproj" (Rebuild target(s)) -- FAILED.
[2023-11-27 22:01:46] [build-stdout] Done Building Project "C:\helloworld\HelloWorld.sln" (rebuild target(s)) -- FAILED.
[2023-11-27 22:01:46] [build-stdout] Build FAILED.

Reproduction steps
For reproduction, a standard Visual Studio HelloWorld template project was used. Latest stable vs_buildtools (17.8.1) and latest CodeQL release (2.15.3) was installed in a docker container (based on mcr.microsoft.com/windows/servercore:ltsc2022)

An attempt to create a new database was done with the command:
C:\devtools\CodeQL\codeql.2.15.3\codeql.exe database create -v -v -v .\HelloWorld\codeql --language=cpp --overwrite --command="msbuild /t:rebuild /p:Configuration=Release /p:Platform=x64 ${PWD}\HelloWorld.sln"

This resulted in the output in the attached logs.

For reference, the output of cl.exe --version is

Microsoft (R) C/C++ Optimizing Compiler Version 19.38.33130 for x86
Copyright (C) Microsoft Corporation.  All rights reserved.

cl : Command line warning D9002 : ignoring unknown option '--version'
cl : Command line error D8003 : missing source filename

The exact same steps with Visual Studio/vs_buildtools 17.4 works flawlessly.

[jketema]

Hi @matlimatli,

Thanks for raising this issue. It seems that we no longer capture the output of cl.exe and it goes to the console instead. We'll investigate.

[jketema]

We believe we have fixed this. The fix should be part of CodeQL 2.15.4, which should be released a few days from now.

[matlimatli]

@jketema Sounds great! However, since HLK testing requires CodeQL 2.6.3, this might still be a problem in some cases. I'm not sure where that issue should be raised.

[jketema]

HLK testing issues should be raised with Microsoft. From our perspective 2.6.3 is no longer supported.

[jketema]

Closing as this was fixed. I believe Microsoft also has put into place some mitigating measures for HLK.