multicore test suite

[cscheid]

tl;dr

$ ./run-parallel-tests.sh
Will run in 10 cores
Expected speedup: 5.89
* The library is already synchronized with the lockfile.

A parallel test suite should be as simple as deno test --parallel, but our test suite is somehow still not working great in that scenario, so this approach takes a bit of extra work. Every time a new test is added (or every time we want to update the timing file because we think the relative runtimes have significantly changed), we need to collect a full slow run.

That's done by QUARTO_TEST_TIMING=1 ./run-tests.sh, which will recreate the tests/timing.txt file.

With that file in place, ./run-parallel-tests.sh uses a simple scheduler to run our test suite across a large number of parallelized individual ./run-tests.sh call.

The report is also a little uglier, but it's functional enough: there's a global "test failed" output if any of the tests fail, and then you have to scroll up to find the test.

A 6x speedup seems worth the effort, though. There's still a largeish improvement to be done by splitting smoke/smoke-all.test.ts as a special case, but this will get us going.

[cscheid]

One of the issues here is that we seem to have different tests attempting to render the same file and then check different things with it. These tests, of course, cannot run in parallel in the same directory because they stomp on each other.

[cscheid]

Simple example: docs/test.Rmd. It is rendered on at least 3 separate tests, all of which have shown up in failures in the parallel test.