Is there any way to tell a github workflow's autobuilder to use parallel make?

[ryao]

The short version:

Is there some way for people to specify in the github workflow that the autobuilder should do a parallel build, or do I need to implement the individual steps myself to get that?

The long version:

At the moment, we are using the autobuilder in a github workflow to do builds:

https://github.com/openzfs/zfs/blob/master/.github/workflows/codeql.yml

By default, the buildsystem only does a single make job unless told otherwise (e.g. via make -j$(nproc)). I am new to both CodeQL and github workflows, so I deemed this to be good enough when I made that commit, but now I am experimenting with additional queries and that nearly doubles the time that it takes to get results:

openzfs/zfs@15e39c6

The autobuilder reminds me of the internals of Gentoo's package management, which similarly has automation so that package maintainers do not have write custom build steps. It also supports an end user passing -j$SOMETHING to the build system via MAKEOPTS, so that package maintainers do not explicitly need to implement support for parallel builds into packaging when the existing build systems already support them. In this same vein, it makes me suspect that there should be a way to declare that I want a parallel build without requiring me to manually write all of the build steps.

Is there some way for people to specify in the github workflow that the autobuilder should do a parallel build, or do I need to implement the individual steps myself to get that?

[aibaars]

Having this configurable for the "autobuilder" is an interesting idea. Thanks!

In the short term, have you tried setting MAKEFLAGS ; if I am not mistake GNU make picks up flags defined in that environment variables. You can set environment variables in a workflow using the env: property.

[ryao]

In a similar vein, Gentoo has a way for end users to pass configure arguments via EXTRA_ECONF, although use of it is discouraged since Gentoo has a more user friendly way of setting known configure arguments. If github were to implement configurability in the autobuilder, that could also include something that is EXTRA_ECONF-like where the flags passed through it would be appended to the end of the configure script invocation.

That said, I will try using MAKEFLAGS with the env property and report back with the result.

[ryao]

That worked, although I hard coded -j2 since I was not certain that -j$(nproc) would work. I will probably test that at some point.

Anyway, -j2 reduced the autobuild time from 27m 13s to 16m 49s, which is a nice improvement. I recall reading somewhere that the builders only had two CPUs. If that is true, then going higher probably would not make a difference, but I can test that. :)

Edit: There are definitely only 2 vCPUs in the runners. I added this to my yml file and then looked at the env output by the autobuilder. It printed 2, so SMT is not available in the runners.

     steps:
+    - name: Set make jobs
+      run: |
+        echo "MAKEFLAGS=-j$(nproc)" >> $GITHUB_ENV
+