New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Replace odd -j in modules-submake with ability for proper parallelism. #128
Conversation
Recently `-j` was added to the list of flags passed to the modules $(MAKE) command. That's bad practise for two reasons: (1) the caller should decide on how much -j is required, not the programmer, and (2) just -j without any numeric argument is "use all resources you can get", which can be too much. We order the Makefile around so the supermake (the user invocation) can dispatch individual submakes in a parallel fashion. This is done by (a) making the module directories phony targets and (b) replacing the sequential for-loop over the modules with a simple dependency.
Sorry for the duplicate. My first pull request was off master. I now force-updated the repo and moved that patch to this branch. Bogdan already commented. He said: Hi Walter, I agree with you said, nevertheless compiling all modules in the same time may be scary and unmanageable - you get the compiling logs mixed all together (from all modules) and in case of compile failure , hard to figure out what module the file belongs to :(... I would suggest a default "-j3" which can be override by user and a super-fast compile mode based on your patch (by default off) triggered via an extra switch or so.... |
Ok. So I took your concerns, and did the following:
Non-scientific tests says, before:
.. and after:
|
I added a new Makefile switch, called FASTER, that acts as a switch for this commit. For more information, see: http://www.opensips.org/Documentation/Install-CompileAndInstall-1-11#toc6 |
Recently
-j
was added to the list of flags passed to the modules$(MAKE) command. That's bad practise for two reasons: (1) the caller
should decide on how much -j is required, not the programmer, and (2)
just -j without any numeric argument is "use all resources you can
get", which can be too much.
We order the Makefile around so the supermake (the user invocation)
can dispatch individual submakes in a parallel fashion. This is done
by (a) making the module directories phony targets and (b) replacing
the sequential for-loop over the modules with a simple dependency.
And, to top it off, I ran a quick and dirty non-scientific test (*) and came up with these tests results:
Before patch, make modules -j3:
real 1m23.744s
user 2m24.796s
sys 0m7.252s
After patch, make modules -j3:
real 1m2.799s
user 2m51.112s
sys 0m7.400s
That's quite a bit of speedup too. Even though that wasn't the goal.
(*) I ran the test twice to rule out caching issues in favor of either one of them and came up with pretty much the same numbers both times.