Perform CodeQL Analysis fails during database finalize due to NoSuchFileException

[makubacki]

While enabling CodeQL, I encounter this error frequently. Occassionally, I do not encounter the error and the GitHub action succeeds.

/opt/hostedtoolcache/CodeQL/0.0.0-20221010/x64/codeql/codeql database finalize --finalize-dataset --threads=2 /home/runner/work/_temp/codeql_databases/cpp --ram=5920
  Running TRAP import for CodeQL database at /home/runner/work/_temp/codeql_databases/cpp...
  A fatal error occurred: Couldn't read /home/runner/work/_temp/codeql_databases/cpp/trap/cpp/source/tmp/ccMtRh��.link
  (eventual cause: NoSuchFileException "/home/runner/work/_temp/codeql_databases/cpp/trap/cpp/source/tmp/ccMtRh��.link")
  A fatal error occurred: Dataset import for /home/runner/work/_temp/codeql_databases/cpp/db-cpp failed with code 2.
  Error: The process '/opt/hostedtoolcache/CodeQL/0.0.0-20221010/x64/codeql/codeql' failed with exit code 2
  Error: The process '/opt/hostedtoolcache/CodeQL/0.0.0-20221010/x64/codeql/codeql' failed with exit code 2
      at toolrunnerErrorCatcher (/home/runner/work/_actions/github/codeql-action/v2/lib/toolrunner-error-catcher.js:86:19)
      at processTicksAndRejections (node:internal/process/task_queues:96:5)
      at async Object.finalizeDatabase (/home/runner/work/_actions/github/codeql-action/v2/lib/codeql.js:[60](https://github.com/tianocore/edk2/actions/runs/3362451486/jobs/5574191203#step:12:61)7:13)
      at async finalizeDatabaseCreation (/home/runner/work/_actions/github/codeql-action/v2/lib/analyze.js:114:13)
      at async runFinalize (/home/runner/work/_actions/github/codeql-action/v2/lib/analyze.js:298:21)
      at async run (/home/runner/work/_actions/github/codeql-action/v2/lib/analyze-action.js:164:29)
      at async runWrapper (/home/runner/work/_actions/github/codeql-action/v2/lib/analyze-action.js:235:9)

The filename that can't be read will change, but the error remains:

A fatal error occurred: Couldn't read /home/runner/work/_temp/codeql_databases/cpp/trap/cpp/source/home/runner/work/edk2/edk2/Build/MdeModule/DEBUG_GCC5/IA32/MdeModulePk��.link

• Failure Example 1
• Failure Example 2
• Successful Example

Can you please help provide more information about what is going on?

[Erich-McMillan]

I'm seeing this issue as well, but locally in an ubuntu container:

Finalizing database at /edk2/codeql-cpp-analysis.
A fatal error occurred: Couldn't read /edk2/codeql-cpp-analysis/trap/cpp/source/�♠.link
(eventual cause: NoSuchFileException "/edk2/codeql-cpp-analysis/trap/cpp/source/�♠.link")
Running queries.
A fatal error occurred: /edk2/codeql-cpp-analysis/db-cpp does not seem to be a raw QL dataset; it has no dbscheme.

[makubacki]

We compile multiple "packages" of code (sets of drivers) with a single build command. Building each package in a separate job with a matrix is leading to better results so far.

https://github.com/tianocore/edk2/actions/runs/3366104711

[makubacki]

While it is easier to rerun an individual failed job with the matrix, the same failure does occasionally occur.

[jketema]

Hi @makubacki

Thanks for opening this issue.

Can you please help provide more information about what is going on?

It seems that your build environment is triggering a bug in the CodeQL component ("extractor") responsible for reading your C/C++ source files. We are able to reproduce this behaviour on the GitHub provided runners, but unfortunately not locally. It's also not a failure we have seen before, while this code has not been modified for the last few months.

Could you tell me whether your build environment does anything special with environment variables that are being set (the extractor depends on a few of these, which are set by the action) or whether you somehow override functionality of the system provided glibc library?

[makubacki]

Hi. I triggered a new workflow to periodically dump environment variables here - https://github.com/tianocore/edk2/actions/runs/3388710334. It has a couple failures that reproduce the issue.

I'm not aware of any overrides to glibc.

[jketema]

Thanks. I don't see anything that is particularly of interest in that output.

I'll file an internal issue for this, so we can schedule looking further into this.

[makubacki]

Great, thanks! It looks like a viable workaround for my project is to use Windows builds. So far, I'm seeing reliable results with that. We are still very interested in being able to build on Ubuntu + GCC though.

• https://github.com/tianocore/edk2/actions/runs/3389189906
• https://github.com/tianocore/edk2/actions/runs/3389380398