Use OCaml code to find the oldest commit

[talex5]

Before, we spawned a separate git log process after each solve. This has the advantage that it can do work independently of the OCaml GC, but it also has some disadvantages:

• git has to read and decompress the repository for each solve, rather than caching it.
• Spawning a process requires briefly pausing all domains, slowing down solving.
• The spawned processes might run on a CPU that OCaml wants to use, delaying that domain and, in turn, disrupting the STW minor GC.

Just looking at the time to find the commit, I get:

Before After
0.035s 0.246s
0.186s 1.727s
0.039s 0.000s
0.038s 0.000s
0.038s 0.000s
0.037s 0.001s
0.039s 0.000s
0.040s 0.000s
0.040s 0.001s
...

That is, the OCaml code is a lot slower at loading the data from disk, but once cached it's massively faster.

For total times (running bench.sh, which does a warm-up first and so excludes loading from disk):

My machine:

jade-2 (with SCHED_RR):

[talex5]

Sample traces for 24 cores

Before:

with_pipe shows when the main domain spawns the git subprocess. Notice that minor GCs stop happening during this, indicating that all the other domains are stuck too. The other two gaps in the GCs are also caused by other fibers spawning subprocesses (not shown here to save space).

After:

Worker domains continue to perform minor GCs, indicating that work is continuing.