feat!: support multiple parallel child_process

[AriPerkkio]

Description

• Many projects are unable to use node:worker_threads and are forced to use a single node:child_process via --no-threads. That's stable, but slow. This PR allows using multiple node:child_process for running tests similarly as --threads does.
  • Hitting Segmentation fault when running Vitest . #3143
  • Vitest crashes when running tests with Prisma #3106
  • Getting Node low level errors running vitest #3816
  • Crashes bcrypt tinylibs/tinypool#48
• Ref. Feature: Support child_process runtime tinylibs/tinypool#64
• Continues in feat!: add support for pool and poolOptions #4172

BREAKING CHANGES

useFakeTimers() no longer automatically mocks process.nextTick. Users can still mock it by explicitly specifying useFakeTimers({ toFake: ['nextTick'] }).

Mocking process.nextTick by using useFakeTimers({ toFake: ['nextTick'] }) is no longer supported when running in --no-threads. Node's child_process uses process.nextTick internally and it is really difficult to support this use case. Users can still mock process.nextTick when running with --threads.

--no-threads no longer runs tests in the same test context. Tests are now isolated in separate processes similar to how --threads does with worker_threads.

Testing

• pnpm test:ci:no-threads
• Manual testing with third party projects:
  • zustand: Tests require isolation so comparing speed change doesn't make sense here. All tests pass with new isolated --no-threads.
  • vuejs/core. Uses --no-threads on CI. Speed improvement 48.57s -> 26.96s.
  • netlify/cli. Tests pass with --no-threads.
  • lerna-lite. Tests require child_process and have been using --no-threads. Speed improvement 67s -> 24s.

---

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

• It's really useful if your PR references an issue where it is discussed ahead of time. If the feature is substantial or introduces breaking changes without a discussion, PR might be closed.
• Ideally, include a test that fails without this PR but passes with it.
• Please, don't make changes to pnpm-lock.yaml unless you introduce a new test example.

Tests

• Run the tests with pnpm test:ci.

Documentation

• If you introduce new functionality, document it. You can run documentation with pnpm run docs command.

Changesets

• Changes in changelog are generated from PR name. Please, make sure that it explains your changes in an understandable manner. Please, prefix changeset messages with feat:, fix:, perf:, docs:, or chore:.

[stackblitz]

Run & review this pull request in StackBlitz Codeflow.

[netlify]

✅ Deploy Preview for fastidious-cascaron-4ded94 ready!

Name	Link
🔨 Latest commit	e05d466
🔍 Latest deploy log	https://app.netlify.com/sites/fastidious-cascaron-4ded94/deploys/65031bc2d89a4a00086fad07
😎 Deploy Preview	https://deploy-preview-3925--fastidious-cascaron-4ded94.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[AriPerkkio]

Seeing some process hangs when running Vitest's test cases with --no-threads. Debugging these is really time consuming and I've spent quite some time looking into this. I'm starting to think we should implement graceful worker_thread and child_process termination in Tinypool. That would allow us to implement Jest's --detect-open-handles feature too.

These errors come up locally and on CI:

⎯⎯⎯⎯⎯⎯ Unhandled Error ⎯⎯⎯⎯⎯⎯⎯
Error: [birpc] timeout on calling "onTaskUpdate"
 ❯ Timeout._onTimeout ../../packages/vitest/dist/vendor-index.9acf798a.js:38:22
 ❯ listOnTimeout node:internal/timers:569:17
 ❯ processTimers node:internal/timers:512:7

[sheremet-va]

Debugging these is really time consuming and I've spent quite some time looking into this.

Welcome to my world! For me it's almost always something related to timer mocks (nextTick included).

I'm starting to think we should implement graceful worker_thread and child_process termination in Tinypool.

Will it help? Or just to debug?

[AriPerkkio]

Will it help? Or just to debug?

Mostly for debugging. But still, I'm not sure if it's good practice to terminate workers and processess with .kill() and .terminate() when there can still be all kind of stuff running, e.g. chokidar with fsevents. I'm starting to think that it's causing some of these process hangs but don't really have proof for it.

The way how jest-worker does this is that main thread instructs thread/process (worker) to remove all its message handlers so that there is nothing keeping the worker running, and then it waits for the worker to gracefully exit. If it doesn't exit in specific timeout, main threads sends it SIGTERM. If it still doesn't exit, it sends SIGKILL. What Tinypool currently does is that it immediately sends SIGTERM and then hangs if worker doesn't exit - or throws timeout error.

But if Vitest used graceful termination for workers, it could impact in performance. We would not be killing workers as fast as possible - instead we would be waiting for them to shut down themselves. I'm not sure if this performance issue is real or just speculation. Jest or Vitest's vm-threads don't really suffer from this as isolation doesn't require recycling workers.

Maybe I'll implement --detect-open-handles into Tinypool so that it's optional flag that can be turned on when needed. It would definitely help debugging this PR's issues.

[AriPerkkio]

[...] and implement --single-thread equivalent for child_process. I don't think there's a way to use that without introducing new configuration option but let's leave that to another PR.

Looks like I already implemented this. It's available with vitest --no-threads --single-thread. The --no-thread makes Vitest pick pools/child.ts and --single-thread makes it run in a single process.

TODO

• Reporters work weird in test/public-api and test/stacktraces when run with --no-threads. Enabling CI=true fixes this.

Looks like this bug is present on main too. When running Vitest through JS API (startVitest()) with --no-threads, the reporters are not resolved correctly. This can be tested with:

cd vitest/test/public-api

# Pass
pnpm run test --threads --run
CI=true pnpm run test --no-threads --run

# Fail
pnpm run test --no-threads --run

[ghiscoding]

so is this complete? on which milestone should we expect this to land, in v0.35 or v1.0? As you can see I'm very exited about it 😄

[sheremet-va]

so is this complete? on which milestone should we expect this to land, in v0.35 or v1.0? As you can see I'm very exited about it 😄

The next major is 1.0.0-beta.0

[ghiscoding]

The next major is 1.0.0-beta.0

@sheremet-va ok but just so you know, you still have a v0.35 milestone that exist with 1 PR assign to it 😋
https://github.com/vitest-dev/vitest/milestone/11

awesome work guys

[ghiscoding]

Many projects are unable to use node:worker_threads and are forced to use a single node:child_process via --no-threads. That's stable, but slow. This PR allows using multiple node:child_process for running tests similarly as --threads does.

• Hitting Segmentation fault when running Vitest . #3143
• Vitest crashes when running tests with Prisma #3106
• Getting Node low level errors running vitest #3816

@AriPerkkio I'm just curious, were the issues you referenced supposed to be fixed (aka closed) since this PR got merged? Thanks a ton for the PR on my side 😃

[AriPerkkio]

Yes, I think we should close those now. These changes are the best solution we can offer for cases where node:worker_threads simply won't work with users code.

I'll go through those later today and leave a comment describing the changes while closing them.