Skip to content

v1.4.1

Choose a tag to compare

@noderaven noderaven released this 10 Oct 06:01
d286e80

This release makes the test suite run its files in parallel. Nothing in the skill's runtime behaviour changes.

Highlights

  • New test runner: node tests/run-tests.mjs. It names every tests/*.test.mjs to node --test, so each file runs in its own process, four at a time by default (PL_TEST_CONCURRENCY or --concurrency N to change it; other arguments go to node --test). It exits with node's exit code, and 2 on a usage error. (#15, #16)
  • Why it is faster: node --test tests/ loads tests/index.js, which imports every test file one after another in a single process, so the suite never ran files in parallel. That command still works, but the runner is now the documented one, and CI uses it.
  • No silent passes when nested: node --test sets NODE_TEST_CONTEXT in the processes it starts, and a node --test that inherits it runs nothing and exits 0. The runner removes it, so a run started from inside a test process still runs and reports failures.

Measurements

Windows 11 (20 cores), Node 24.17.0, the v1.4.0 suite (754 tests). Every run passed with the same counts and left nothing in the checkout.

One suite alone, python3 through the app alias (see Notes):

Command Runs (s)
node --test tests/ 266.6, 268.2, 268.8
one file at a time, separate processes 272.1, 273.8, 278.2
two at a time 147.1, 147.2, 147.7
four at a time 93.4, 94.8, 96.6

After putting Python\bin first on PATH: one suite alone took 130.0, 130.7, 130.8 s with two at a time and 96.4, 97.2, 97.3 s with four. Four suites at once in separate checkouts, as four lanes run them (one round): 299.1-300.6 s per suite through node --test tests/, 214.4-216.2 s with two at a time, 200.2-200.7 s with four.

Testing

  • node tests/run-tests.mjs, 761 tests, 0 failures everywhere. CI: Ubuntu (Node 22 and 24) 759 pass, 2 skipped, about 20 s; macOS 760 pass, 1 skipped, 45 s; Windows 749 pass, 12 skipped, 121 s (about 243 s through node --test tests/ before #15). Every job also checks that the suite leaves the checkout unchanged.
  • tests/run-tests.test.mjs (7 tests) runs the runner against fixture files: one process per file, the default and overridden concurrency, failures passed through, option pass-through, a nested start, usage errors. The default test failed before each default change.

Notes

  • Windows, python3 through the app alias: if python3 resolves to %LOCALAPPDATA%\Microsoft\WindowsApps\python3.exe (the Python install manager's app execution alias), each start goes through Windows app activation. On one machine that cost about 55 ms per start (76 ms against 21 ms through %LOCALAPPDATA%\Python\bin\python3.exe, the same interpreter), and the suite starts Python several hundred times. Putting %LOCALAPPDATA%\Python\bin before WindowsApps on PATH avoids it. On that machine the two-at-a-time median fell from 147 s to 131 s after the change (separate sessions, not a controlled comparison).
  • Known issue: two checks in tests/install.test.mjs start sh by name, so the suite fails there when started from PowerShell or cmd, where sh is not on PATH. Run it from Git Bash (as before) until this is fixed.

Upgrade

In your clone:

git pull && bash install.sh

Then run the tests with node tests/run-tests.mjs.

Measurement clarification (edited after release): the 147 s to 131 s figure is a before/after observation from separate sessions, not a controlled comparison of the two PATH orders.