Performance with many test files is showstopper #1631
Description
In my projects I have individual test files per component, resulting in hundreds of small test files. The performance in this scenario is a showstopper for using AVA. The issue is the cost of forking node processes as recognised in #789. I hoped it might be useful to gather some stats to show the scale of the issue. I created a little script to auto-generate test files and ran them against AVA, Mocha and Jest
| Files * Tests per file | AVA | Mocha | Jest |
|---|---|---|---|
| 7 * 1428 | 2.8 (10.2) | 4.2 (72) | 4.0 (26.0) |
| 8 * 1250 | 2.9 (10.1) | 4.1 (53) | 4.1 (24) |
| 10 * 1000 | 3.5 (10.6) | 4.2(37) | 4.2 (15.6) |
| 100 * 100 | 17.2 (25.3) | 5.5 (12.7) | 5.4 (8.3) |
| 500 * 20 | 73 (78) | 4.8(10.3) | 7.5 (10.3) |
In each case total number of tests executed is 10K - split across different number of files
All times are in seconds
Time in brackets is first run. Presumably higher because of Babel transpilation.
Test is
import test from 'ava';
const double = (x) => x * 2;
test('test double 1', t => {
t.is(double(1), 2);
});
...
As you can see AVA goes from being significantly faster :-) than the others to an order of magnitude slower dependent on the number of files :-(
I was testing on 8 core MacBook Pro running OS X 10.10.5.
NodeJs: 8.6.0
AVA: 0.24
Any chance of upping the priority of #789 ?