Is there interest in replacing mocha with node:test

[cjihrig]

Since v18, Node has shipped a built in test runner, which we could replace mocha with. This is what the change would look like.

Pros:

• Fewer dependencies, which have been causing us grief lately. Mocha hasn't been problematic, and is probably my second favorite test runner. However, mocha still weighs in at 8.5MB and 104 dependencies.
• Better security posture because it is built in.
• node:test seems to be better at reporting things that mocha skips over like unhandled rejections from not awaiting assert.rejects().
• For what we do in this project, the required API is essentially the same.
• It also has built in code coverage, although that is still experimental, mocking, snapshots, etc.

Cons:

• The biggest con is speed. In my testing, node:test and mocha have pretty similar speed when run in the same mode. Unfortunately, Node's "fast" mode (running all test files in the same process) only exists in v22 and up. Since we test on v18 and v20, we would have to use process level isolation for each test file, which adds a bit of overhead, although I don't think it's enough to really be noticeable. Fun fact: this is how I learned that we were using chai-as-promised inconsistently.
• Because it was only introduced in v18, it has been less stable between versions. From my testing, none of the changes between versions seem to impact this project other than having to use process level isolation.

A number of notable Node projects have adopted node:test, so I don't think this would mean using something immature.

Disclaimer: I wrote node:test, so I am definitely biased here. If we don't want to migrate, no hard feelings.

[brendandburns]

I'm definitely open to migrating. Can we get still get the coverage data easily if we switch to node:test

wrt to slowing down tests on 18 & 20 I don't think that really matters since people doing development should probably be on 22 and the 18/20 tests are only for CI/CD where speed is a little less important.

[cjihrig]

Can we get still get the coverage data easily if we switch to node:test

Yes, c8 still works exactly the same.

wrt to slowing down tests on 18 & 20

We would be doing this for all release lines until Node 20 is EOL in April 2026. The alternative would be having different npm test scripts depending on the version, but I don't think it's worth the added complexity. For more concrete numbers, using Node v23.10.0, time npm test takes just over 6 seconds with node:test on my machine, while mocha currently takes about 3 seconds. I did try running the tests in the same process, and node:test was equal to or a bit faster than mocha, but as I mentioned, we'd have to wait to leverage that feature on all release lines.

[brendandburns]

That seems totally fine. That said, it's easy enough to write a bash script that detects the version of node and adds/removes flags as needed. That way it's the same configuration for all versions, but it runs differently depending on the environment.

So let's go forward with this, we can adapt the script if we want to.