codeql stuck for 30+ minutes during FindHDF5.cmake execution

[rouault]

On a Ubuntu 20.04 host with the following CMakeLists.txt:

cmake_minimum_required(VERSION 3.16)
project(gdal LANGUAGES C CXX)
find_package(HDF5 COMPONENTS C CXX)

and package libhdf5-dev installed

running "cmake .." takes more than half an hour

Verbose output:

$ cmake .. --debug-output
Running with debug output on.
-- The C compiler identification is GNU 9.4.0
   Called from: [3]	/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake
                [2]	/usr/share/cmake-3.16/Modules/CMakeDetermineCCompiler.cmake
                [1]	/home/even/gdal/gdal/build_cmake/tmptmphdf5/CMakeLists.txt
-- The CXX compiler identification is GNU 9.4.0
   Called from: [3]	/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerId.cmake
                [2]	/usr/share/cmake-3.16/Modules/CMakeDetermineCXXCompiler.cmake
                [1]	/home/even/gdal/gdal/build_cmake/tmptmphdf5/CMakeLists.txt
-- Check for working C compiler: /usr/bin/cc
   Called from: [3]	/usr/share/cmake-3.16/Modules/CMakeTestCompilerCommon.cmake
                [2]	/usr/share/cmake-3.16/Modules/CMakeTestCCompiler.cmake
                [1]	/home/even/gdal/gdal/build_cmake/tmptmphdf5/CMakeLists.txt
-- Check for working C compiler: /usr/bin/cc -- works
   Called from: [3]	/usr/share/cmake-3.16/Modules/CMakeTestCompilerCommon.cmake
                [2]	/usr/share/cmake-3.16/Modules/CMakeTestCCompiler.cmake
                [1]	/home/even/gdal/gdal/build_cmake/tmptmphdf5/CMakeLists.txt
-- Detecting C compiler ABI info
   Called from: [3]	/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerABI.cmake
                [2]	/usr/share/cmake-3.16/Modules/CMakeTestCCompiler.cmake
                [1]	/home/even/gdal/gdal/build_cmake/tmptmphdf5/CMakeLists.txt
-- Detecting C compiler ABI info - done
   Called from: [3]	/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerABI.cmake
                [2]	/usr/share/cmake-3.16/Modules/CMakeTestCCompiler.cmake
                [1]	/home/even/gdal/gdal/build_cmake/tmptmphdf5/CMakeLists.txt
-- Detecting C compile features
   Called from: [3]	/usr/share/cmake-3.16/Modules/CMakeDetermineCompileFeatures.cmake
                [2]	/usr/share/cmake-3.16/Modules/CMakeTestCCompiler.cmake
                [1]	/home/even/gdal/gdal/build_cmake/tmptmphdf5/CMakeLists.txt
-- Detecting C compile features - done
   Called from: [3]	/usr/share/cmake-3.16/Modules/CMakeDetermineCompileFeatures.cmake
                [2]	/usr/share/cmake-3.16/Modules/CMakeTestCCompiler.cmake
                [1]	/home/even/gdal/gdal/build_cmake/tmptmphdf5/CMakeLists.txt
-- Check for working CXX compiler: /usr/bin/c++
   Called from: [3]	/usr/share/cmake-3.16/Modules/CMakeTestCompilerCommon.cmake
                [2]	/usr/share/cmake-3.16/Modules/CMakeTestCXXCompiler.cmake
                [1]	/home/even/gdal/gdal/build_cmake/tmptmphdf5/CMakeLists.txt
-- Check for working CXX compiler: /usr/bin/c++ -- works
   Called from: [3]	/usr/share/cmake-3.16/Modules/CMakeTestCompilerCommon.cmake
                [2]	/usr/share/cmake-3.16/Modules/CMakeTestCXXCompiler.cmake
                [1]	/home/even/gdal/gdal/build_cmake/tmptmphdf5/CMakeLists.txt
-- Detecting CXX compiler ABI info
   Called from: [3]	/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerABI.cmake
                [2]	/usr/share/cmake-3.16/Modules/CMakeTestCXXCompiler.cmake
                [1]	/home/even/gdal/gdal/build_cmake/tmptmphdf5/CMakeLists.txt
-- Detecting CXX compiler ABI info - done
   Called from: [3]	/usr/share/cmake-3.16/Modules/CMakeDetermineCompilerABI.cmake
                [2]	/usr/share/cmake-3.16/Modules/CMakeTestCXXCompiler.cmake
                [1]	/home/even/gdal/gdal/build_cmake/tmptmphdf5/CMakeLists.txt
-- Detecting CXX compile features
   Called from: [3]	/usr/share/cmake-3.16/Modules/CMakeDetermineCompileFeatures.cmake
                [2]	/usr/share/cmake-3.16/Modules/CMakeTestCXXCompiler.cmake
                [1]	/home/even/gdal/gdal/build_cmake/tmptmphdf5/CMakeLists.txt
-- Detecting CXX compile features - done
   Called from: [3]	/usr/share/cmake-3.16/Modules/CMakeDetermineCompileFeatures.cmake
                [2]	/usr/share/cmake-3.16/Modules/CMakeTestCXXCompiler.cmake
                [1]	/home/even/gdal/gdal/build_cmake/tmptmphdf5/CMakeLists.txt
-- HDF5: Using hdf5 compiler wrapper to determine C configuration
   Called from: [2]	/usr/share/cmake-3.16/Modules/FindHDF5.cmake
                [1]	/home/even/gdal/gdal/build_cmake/tmptmphdf5/CMakeLists.txt

Relevant processes that take ages to complete:

even     3102981 3694545  1 02:30 pts/8    00:00:00 cmake .. --debug-output
even     3103156 3102981  0 02:30 pts/8    00:00:00 /bin/sh /usr/bin/h5c++ -show -shlib /home/even/gdal/gdal/build_cmake/tmptmphdf5/build/CMakeFiles/hdf5/cmake_hdf5_test.cxx
even     3103177 3103156 11 02:30 pts/8    00:00:01 /home/even/gdal/gdal/build_cmake/codeql/cpp/tools/linux64/extractor --mimic /usr/bin/h5c++ -show -shlib /home/even/gdal/gdal/build_cmake/tmptmphdf5/bui
even     3103178 3103177  0 02:30 pts/8    00:00:00 /bin/sh /usr/bin/h5c++ -x c++ -v -fsyntax-only /home/even/gdal/gdal/build_cmake/codeql_databases/cpp/working/tmp//semmle_1_3103177_195397
even     3103179 3103178  0 02:30 pts/8    00:00:00 /bin/sh /usr/bin/h5c++ -x c++ -v -fsyntax-only /home/even/gdal/gdal/build_cmake/codeql_databases/cpp/working/tmp//semmle_1_3103177_195397
even     3103198 3103179  0 02:30 pts/8    00:00:00 g++ -I/usr/include/hdf5/serial -x c++ -v -fsyntax-only /home/even/gdal/gdal/build_cmake/codeql_databases/cpp/working/tmp//semmle_1_3103177_195397 -L/us
even     3103199 3103198  0 02:30 pts/8    00:00:00 g++ -I/usr/include/hdf5/serial -x c++ -v -fsyntax-only /home/even/gdal/gdal/build_cmake/codeql_databases/cpp/working/tmp//semmle_1_3103177_195397 -L/us
even     3103224 3103199  0 02:30 pts/8    00:00:00 /usr/lib/gcc/x86_64-linux-gnu/9/cc1plus -quiet -v -I /usr/include/hdf5/serial -imultiarch x86_64-linux-gnu -D_GNU_SOURCE /usr/lib/x86_64-linux-gnu/hdf5
even     3103225 3103224  0 02:30 pts/8    00:00:00 /usr/lib/gcc/x86_64-linux-gnu/9/cc1plus -quiet -v -I /usr/include/hdf5/serial -imultiarch x86_64-linux-gnu -D_GNU_SOURCE /usr/lib/x86_64-linux-gnu/hdf5

(originally diagnosed in OSGeo/gdal#9549)

[jketema]

Hi @rouault,

It's possible hanging in the extractor process, but from the information you provide it's not easy to tell/debug further. My recommendation would be to run cmake before you initialise CodeQL. So you'll get a workflow which roughly has the following steps:

1. checkout
2. install dependencies
3. run cmake
4. initialise CodeQL
5. build
6. analyse

Looking at your workflow, it seems it's also using ccache. Note that CodeQL is not compatible with ccache. The workflow will need to build the whole project from scratch to obtain a complete database and complete analysis results.

I also get the impression that the code that is being built might depend on the which CPU features the present on the runner being used. If this means that some code is built on one runner, but not on another, then this might lead to wobble in the alerts you get, because different code will be seen during different analysis runs.

[echoix]

@jketema if you open the linked issue, where I originally reported it, I had created a complete repro that was possible to reproduce locally, in a devcontainer, with all steps scripted.

Your suggestion about ccache makes sense. I also had thought of your other suggestion of configuring CMake before starting CodeQL, but wasn't sure what CodeQL needs to see when it intercepts the build. So you suggesting it tells me that there isn't any info in these steps that CodeQL uses. But at the same time, that only works around the issue, there is still a long hang happening and not understood yet.

[jketema]

Understood. Unfortunately, debugging this further is not something we can give high priority, in part because we have gotten very few similar reports.