Skip to content

emmake breaks parallel make with older GNU make #25450

@JonathanLennox

Description

@JonathanLennox

Please include the following in your bug report:

Version of emscripten/emsdk:

clang version 22.0.0git (https:/github.com/llvm/llvm-project 3388d40684742e950b3c5d1d2dafe5a40695cfc1)
Target: wasm32-unknown-emscripten
Thread model: posix
InstalledDir: /Users/jlennox/Git/ThirdParty/emsdk/upstream/bin

On older versions of GNU make on Posix systems, invoking an emmake-wrapped parallel make in a subdirectory prints an warning and is not parallelized.

(Note that the version of make on macOS systems is the 2006-vintage GNU make 3.81, because Apple doesn't ship GPLv3 code. I encountered this problem on a Mac.)

E.g., running make -j10 on a makefile containing

$(LIBOPUS_WASM_LIB): libopus-configure
    +emmake $(MAKE) -C $(LIBOPUS_BUILD) libopus.la -r

will print

make[1]: warning: jobserver unavailable: using -j1.  Add `+' to parent make rule.

The sub-make is then not parallelized, and so will be slow.

For an explanation of what is going on, see the documentation of the GNU make jobserver at https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html, and also the stackexchange discussion at https://stackoverflow.com/questions/29910944/is-it-possible-to-pass-through-gnu-make-jobserver-environment-to-a-submake-ser .

Specifically, when GNU make uses the "pipe" mode of the jobserver (the older mode, which is the only one supported in make 3.81), the file descriptors it opens need to be passed down through the emmake process to the subsidiary make process.

Adding the close_fds=False parameter to the subprocess.run invocation in run_process will accomplish this, but it's not clear to me if this might have other negative consequences. (However, python before version 3.2 seems to have had this behavior by default, so it might be harmless?)

Alternately, if necessary, emmake could parse the MAKEFLAGS environment variable and pass a pass_fds parameter to subprocess.run if it looks like make is using a pipe-mode jobserver.

I should be able to write a PR for either solution.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions