Support non-standard compilation processes

[mvanotti]

In a recent talk with @adityasharad , he mentioned that CodeQL would try to understand when a compiler is being invoked. Some projects use goma to speed up the build process, reusing previously built artifacts.

CodeQL seems to ignore all the artifacts that are obtained via goma.

[p0]

To process C++ code, we indeed need to know how to compile it. The easiest way of setting that up is codeql's default, but that indeed relies on observing compilations locally, and distributed build systems like goma or bazel will not work with that approach without disabling the "distributed" aspect.

One possible approach is to leverage compile-commands.json files, as generated by many build systems. The information in them is sufficient to drive the CodeQL tooling, but obtaining such a file is a non-standard build-system-specific and sometimes project-specific process. One possibility would be to add support in CodeQL for creating a database based on such compiler settings files. [It's worth noting that this would not be equivalent to tracing a full local build, since CodeQL takes advantage of information from linker invocations too, and those are not represented in compilation command databases.]

[mvanotti]

Maybe it would be good to have a way of determining what is needed for CodeQL to work properly, and then each build system could figure out how to export that data somehow. That way, having extractors for different build systems depends on the community.

[Manouchehri]

Using bear + #9 would be a decent solution in my opinion. =)

[p0]

I would expect CodeQL's built-in support to be able to handle any situation where bear would also work. The problem with goma or bazel is that the compilations end up being done on a server process or different machine, and are invisibile to whatever local monitoring you are trying to do of the build process.

[haxmeadroom]

Has CodeQL been tested to work with bazel by disabling the distributed aspect, or is this hypothetical? What changes were made to bazel to accomplish this?

To process C++ code, we indeed need to know how to compile it. The easiest way of setting that up is codeql's default, but that indeed relies on observing compilations locally, and distributed build systems like goma or bazel will not work with that approach without disabling the "distributed" aspect.

[adityasharad]

For Bazel, one approach that we have used successfully is the following. It can be passed as the build command to codeql database create or use as a run shell step within a GitHub Actions workflow for CodeQL code scanning.

bazel shutdown; bazel build --spawn_strategy=local --nouse_action_cache //path/to/build/targets/...

• shutdown stops all locally-running Bazel servers
• --spawn_strategy=local disables the distributed aspect
• --nouse_action_cache disables the action cache, increasing the likelihood that all your code is recompiled during the build

More involved integration into Bazel's dependency graph is possible but not likely for the majority of use cases. Please try this and let us know if it helps.

[haxmeadroom]

The above approach did not work for me. I've also tried many combinations with --spawn_strategy=local, --nouse_action_cache, --batch, --action_env=LD_PRELOAD=...lib64trace.so, etc... I have fuzzgoat that builds outside of bazel and gets 12 results (in the csv file output). If I build inside bazel, it runs the 142 evaluations but returns no rows in the csv file. I also always get a warning from bazel that LD_PRELOAD is being ignore. My impression is LD_PRELOAD is required to work, right? Any ideas?

[adityasharad]

@haxmeadroom could you share more about the project you're building (link if it's open source) and the build commands you're using with and without Bazel?

[pestophagous]

I discovered CodeQL this weekend (while digging around in GitHub repo settings looking for other things).

I enabled it to see what would happen, but beyond that I have put essentially zero extra time into reconfiguring my build or trying to get CodeQL to work better on my repo.

Relevant to this bug/enhancement ticket:

• My build toolchain is qmake (for now), and it seems that any cc/cpp files built in my qmake build are not being scanned.

My build also uses a submodule pointing to a different project that uses CMake, and when my build first compiles that project (which is a dependency of my app), then those files built with CMake do appear to get scanned. I know this because there are 3 warnings from the submodule codebase.

I'm actually quite pleased to see the scan including the submodule code. (After all, any vulnerabilities in the submodule will become "my" vulns after I link to that library.)

Now I just need the scan to include my code, too!

Sometime (on the weekends, for my weekend-only side project), I am willing to tweak my build script to help the scanning work.

---

QUESTION:

• Where in the Analysis results (in GitHub web UI) or in the CI/Action log can I see a list of all the cc/cpp files that are scanned?

There must be a list (?), so I don't need to keep injecting bad code into files to see if a warning appears.

---

Here is the PR where I investigated that my own code does not trigger CodeQL warnings: pestophagous/heory#46

I injected the same "Multiplication result converted to larger type" issue into my code to match the issue that I saw trigger a warning in the submodule. But the scan result says "No new or fixed alerts".

[adityasharad]

@pestophagous you can see a brief summary of the lines of code seen by CodeQL within your Actions logs here: https://github.com/pestophagous/heory/runs/2765497253?check_suite_focus=true#step:5:257 (Analysis summary for <language>).

We're in the process of rolling out some new features that give you additional diagnostic information about the codebase that was analysed, such as the number of files (or the list of files when running with higher verbosity). Will report back when you can try those out.

[pestophagous]

@adityasharad This is all I see when I follow the link to the "brief summary" that you mention:

Analysis produced the following metric data:

|                  Metric                   | Value  |
+-------------------------------------------+--------+
| Total lines of C/C++ code in the database | 775466 |
##[endgroup]
##[group]Analysis summary for cpp
Counted 605060 lines of code for cpp as a baseline.
Analysis produced the following metric data:

|                  Metric                   | Value  |
+-------------------------------------------+--------+
| Total lines of C/C++ code in the database | 775466 |

That provides a "no" answer to "can I see a list of all the cc/cpp files?"

Right? I'm not mad if the answer is "no". I just want to clearly understand if it is yes or no to make sure I didn't misunderstand or follow an incorrect link.

It's great to hear you are working on additional features! I look forward to it. I contributed to this ticket only in the spirit of "giving back" and providing more real-life test cases for the team. I'm not complaining! (How could I, this is all provided free of cost to me!)

Thanks for your reply and interest.

[mvanotti]

@pestophagous , I created issue #13 to track what you are asking for. There's a query that will give you the list of files that are in the database.