make: Build KF images in parallel #6873
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Currently there is a single build-all rule in the top-level Makefile under components/ dir that calls all sub-makefiles for building all images in a serial manner. This makes the whole proccess very time-consuming as we can't use the "-j" option of make to run jobs in parallel. Changes to the top-level Makefile for building all KF images: - create a single rule for each directory containing a sub-makefile - have each directory rule as a dependency to the build-all rule in the central Makefile Similarly, the central makefile for building all the example-notebook-servers images calls each sub-Makefile for each of the notebook servers in a serial manner. Changes to the central Makefile for building all notebook-server-images: - Split the single target rule into multiple rules which perform recursive make calls (sub-makefiles) to build all the notebook-server-images. - Use the variable MAKE for recursive make commands instead of explicit 'make' command: - https://www.gnu.org/software/make/manual/html_node/MAKE-Variable.html#MAKE-Variable Signed-off-by: Apostolos Gerakaris <apoger@arrikto.com>
apo-ger
force-pushed
the
feature-apoger-build-kf-images-in-parallel
branch
from
8588d7e
to
5c49763
Compare
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: kimwnasptd The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR addresses #6872
Currently there is a single build-all rule in the
top-level Makefileunder components/ dir that calls all sub-makefiles for building all KF images in a serial manner. This makes the whole process very time-consuming as we can't use the-joption ofmaketo run jobs in parallel.Changes to the
top-level Makefilefor building all KF images:This way every directory rule can run in parallel with other rules.
Similarly, the central makefile for building all the
example-notebook-serversimages calls eachsub-Makefilefor each of the notebook servers in a serial manner.Changes to the central Makefile for building all notebook-server-images:
notebook-server-images.MAKEfor recursive make commands instead of explicitmakecommand:Signed-off-by: Apostolos Gerakaris apoger@arrikto.com