Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
Cargo should release Make jobserver token when it's waiting for file lock #6747
Consider the following Makefile:
And the directory containing the Makefile contains the following
Now, run the build with
The timeline of what happens looks like the following:
Obviously, this is all synthetic, and cargo releasing the make jobserver token as proposed in the summary would not change the outcome in this example, being that the build would still take 4 seconds. But what it would allow is for make to start running the third target earlier, and that could make a hell of a difference.
The real world manifestation is https://bugzilla.mozilla.org/show_bug.cgi?id=1533988 which is caused by the Firefox build system invoking three cargo commands in the same workspace, two of which are "Blocking waiting for file lock on build directory", and the one that does work is actually likely to not be using all the cores available because of how a lot of the dependency chain of the crates built is linear. So the two cargos doing nothing are preventing more C++ compiler processes working while the first cargo works.
added a commit
Mar 14, 2019
referenced this issue
Mar 14, 2019
@glandium pinged me on the original bug, so I tried to test that PR on the exactly same environment (a GCP n1-highcpu-8 instance). In conclusion, that patch helps CPU utilization a lot, although it doesn't seem to help on total build time of Firefox at all (which is a separate issue related to Firefox's build system I guess).
As can be seen that with the patched Cargo, CPU utilization is keep being 100% for most of the time, while previously much less.