ocaml: enable parallel building #142723
Closed
ocaml: enable parallel building #142723
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Enable parallel building for ocaml-4.06 and above. tested as:
$ nix build -f. ocaml-ng.ocamlPackages_{4_{00_1,01_0,02,03,04,05,06,07,08,09,10,11,12,13},latest}.ocaml --keep-going --rebuild
ocaml build system supports parallel building, but but for multiple
top-level targets at the same time as it usually spawns subprocess
$(MAKE) that occasionally conflict with one another. To work it around
we use tiny Makefile with a single rule that calls top-level targets
sequentially as makefile calls:
nixpkgs_world_bootstrap_world_opt:
$(MAKE) world
$(MAKE) bootstrap
$(MAKE) world.opt
On a 16-core machine ocaml-4.12 build speeds up from 6m55s to 1m35s.
Releases 4_00_1, 4_01_0, 4_04 and 4_05 still have some race in them.
Thus this change enables parallel builds only for ocaml-4.06 and above.
28 tasks
|
ocaml 4.12.0 builds successfully for me on nixos x86_64 and macos 10.15. Can't easily say whether it actually has any effect on macos though. |
|
cc @vbgl |
|
Are you sure you want to change the derivation for OCaml 4.00.1? |
|
When/if So far it does it by having a verbatim copy of |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: |
symphorien
approved these changes
Dec 9, 2021
|
IMHO, the change made to the derivation for OCaml 4.00 makes no sense. |
13 tasks
|
reopened without the ocaml 4.00 change as #150128 |
symphorien
pushed a commit
to symphorien/nixpkgs
that referenced
this pull request
Feb 27, 2022
Enable parallel building for ocaml-4.08 and above. tested as:
$ nix build -f. ocaml-ng.ocamlPackages_{4_{00_1,01_0,02,03,04,05,06,07,08,09,10,11,12,13},latest}.ocaml --keep-going
ocaml build system supports parallel building, but but for multiple
top-level targets at the same time as it usually spawns subprocess
$(MAKE) that occasionally conflict with one another. To work it around
we use tiny Makefile with a single rule that calls top-level targets
sequentially as makefile calls:
nixpkgs_world_bootstrap_world_opt:
$(MAKE) world
$(MAKE) bootstrap
$(MAKE) world.opt
On a 16-core machine ocaml-4.12 build speeds up from 6m55s to 1m35s.
Releases 4_00_1, 4_01_0, 4_04 and 4_05 still have some race in them.
Thus this change enables parallel builds only for ocaml-4.06 and above.
Adapted from NixOS#142723
upstreams's CI tests the parallel makefile: ocaml/ocaml#10235 (comment)
The limit was chosen to be 4.08 because it was released in 2019, not too
long before the above link.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Enable parallel building for ocaml-4.06 and above. tested as:
ocaml build system supports parallel building, but but for multiple
top-level targets at the same time as it usually spawns subprocess
$(MAKE) that occasionally conflict with one another. To work it around
we use tiny Makefile with a single rule that calls top-level targets
sequentially as makefile calls:
On a 16-core machine ocaml-4.12 build speeds up from 6m55s to 1m35s.
Releases 4_00_1, 4_01_0, 4_04 and 4_05 still have some race in them.
Thus this change enables parallel builds only for ocaml-4.06 and above.