A simple OCaml library for forking child processes to perform work on multiple cores.
ForkWork is intended for workloads that a master process can partition into independent jobs, each of which will typically take a while to execute (several seconds, or more). Also, the resulting values should not be too massive, since they must be marshalled for transmission back to the master process.
Among the numerous tools for multicore parallelism available in the OCaml ecosystem, ForkWork fits somewhere in between Netmcore and Parmap. It's a bit easier to use than the former, and a bit more flexible than the latter.
ForkWork is mainly tested on Linux and Mac OS X with OCaml 3.12 and above. It should work on most flavors of Unix. The easiest way to install it is by using OPAM:
opam install forkwork.
If you don't use OPAM, set up findlib,
OCAMLFIND_DESTDIR environment variable if necessary and
git clone https://github.com/mlin/forkwork.git && cd forkwork && make && make install
You can then use
ocamlfind as usual to include the
forkwork package when
compiling your program (making sure
you changed it).
ForkWork provides a high-level interface consisting of parallel map functions for lists and arrays. Here's a program that forks four worker processes to compute a Monte Carlo estimate of π:
(* Worker: sample k points uniformly at random from the unit square; return how many fall inside and outside of the unit circle *) let worker k = Random.self_init (); let inside = ref 0 in let outside = ref 0 in begin for i = 1 to k do let x = Random.float 1.0 in let y = Random.float 1.0 in incr (if x *. x +. y *. y <= 1.0 then inside else outside) done; (!inside,!outside) end ;; let estimate_pi n k = (* Fork n parallel worker processes to collect samples *) let results = ForkWork.map_array worker (Array.make n k) in (* Combine the results and derive the estimate *) let insides, outsides = List.split (Array.to_list results) in let inside = float (List.fold_left (+) 0 insides) in let outside = float (List.fold_left (+) 0 outsides) in 4.0 *. (inside /. (inside +. outside)) ;; let n = 30 and k = 25_000_000 in Printf.printf "Based on %.1e samples, π ≈ %f\n" (float (n*k)) (estimate_pi n k) ;;
Run this like so:
$ ocamlfind ocamlopt -o estimate_pi -package forkwork -linkpkg estimate_pi.ml && time ./estimate_pi Based on 7.5e+08 samples, π ≈ 3.141497 real 0m51.119s user 3m16.268s sys 0m1.084s
One other salient feature of ForkWork's high-level interface is that it tries to deal with worker exceptions in a reasonable way, which is difficult because exceptions cannot be marshalled reliably. There's a mechanism for workers to cause ForkWork to both abort the parallel computation and also raise an OCaml exception to the caller with specifc information about the problem. The author was motivated to write ForkWork in part because similar existing libraries did not handle this well, at the time.
There is also a lower-level interface providing much more control over the scheduling of child processes and retrieval of their results. For example, the master process can do other things while worker processes are running, including launch more worker processes, and the result of any individual child process can be retrieved as soon as it finishes.
A fancier version of the estimate_pi example uses the lower-level interface to run the Monte Carlo sampling until the estimate reaches a certain theoretical accuracy threshold, using the available processors continuously and terminating the outstanding workers when done.
The tests use kaputt. To run them,
./configure --enable-tests and