vmThreads does not fail CI step when a test fails

[boryscastor]

Describe the bug

I have quite a big project with 400+ test files, sometimes heavy ones, 2500+ test cases. Recently migrated from jest to vitest, and using pool: vmThreads (the default one is super slow).

I explicitly set a test to fail, and the whole GHA step does not fail, though I do see this fail in vitest script. But the job quits with a simple text: Done in 397.30s.

When I run with default pool, I have the whole stats (number of suites, number of tests, etc.)

Strange thing: using vmThreads I don't see all the 4500 lines of the output that I see using the default pool.

I tried multiple configs but I am falling into veeeery slow tests (25 to 31 minutes).

Reproduction

Vite version: 8.0.16
Vitest version: 4.1.8

Script I use: TZ=Europe/Amsterdam vitest run --reporter=default --reporter=junit --outputFile.junit=./build/junit.xml --maxWorkers=2 (the number of workers does not help much; the default one floods the CI runner, =4 is a bit better)

Vitest config:

import path from 'path';
import react from '@vitejs/plugin-react-swc';
import svgr from 'vite-plugin-svgr';
import { defineConfig } from 'vitest/config';

// Stub static assets so snapshots stay stable. Must run before vite-plugin-svgr (enforce: 'pre').
const staticAssetStubPlugin = {
  name: 'static-asset-stub',
  enforce: 'pre' as const,
  resolveId(id: string) {
    if (/\.(svg|gif|ttf|eot|woff2?)(\?.*)?$/.test(id)) {
      return path.resolve('./src/js/test/fileMock.mjs');
    }
  },
};

export default defineConfig({
  plugins: [
    staticAssetStubPlugin,
    react(),
    svgr({
      include: '**/*.svg',
      svgrOptions: {
        exportType: 'default',
      },
    }),
  ],
  css: {
    preprocessorOptions: {
      scss: {
        silenceDeprecations: ['import', 'global-builtin'],
      },
    },
  },
  define: {
    'process.env.NODE_ENV': JSON.stringify('test'),
  },
  resolve: {
    extensions: ['.tsx', '.ts', '.jsx', '.js', '.json'],
    alias: [
      // @castoredc/i18n ships index.cjs.js inside a "type":"module" package — Node
      // treats it as ESM and breaks on `exports`. Alias directly to the ESM build.
      {
        find: '@castoredc/i18n',
        replacement: path.resolve(
          './node_modules/@castoredc/i18n/dist/index.esm.js'
        ),
      },
      // @castoredc/matter's CJS build requires @castoredc/i18n via native require(),
      // bypassing Vite's alias. Alias to the ESM build so Vite intercepts all its
      // imports and the @castoredc/i18n alias above applies.
      // Regex for exact match — string prefix would also catch @castoredc/matter-fonts etc.
      {
        find: /^@castoredc\/matter$/,
        replacement: path.resolve(
          './node_modules/@castoredc/matter/lib/esm/src/index.js'
        ),
      },
      {
        find: '@castoredc/matter-fonts',
        replacement: path.resolve('./vite/mock-matter-fonts.js'),
      },
      { find: 'apps', replacement: path.resolve('./src/js/apps/') },
      { find: 'components', replacement: path.resolve('./src/js/components/') },
      { find: 'config', replacement: path.resolve('./src/js/config/') },
      { find: 'constants', replacement: path.resolve('./src/js/constants/') },
      { find: 'dialogs', replacement: path.resolve('./src/js/dialogs/') },
      { find: 'edcUtil', replacement: path.resolve('./src/js/edcUtil/') },
      { find: 'modules', replacement: path.resolve('./src/js/modules/') },
      { find: 'helpers', replacement: path.resolve('./src/js/helpers/') },
      { find: 'hooks', replacement: path.resolve('./src/js/hooks/') },
      {
        find: 'integrations',
        replacement: path.resolve('./src/js/integrations/index.ts'),
      },
      {
        find: 'queries',
        replacement: path.resolve('./src/js/queries/index.ts'),
      },
      { find: 'pages', replacement: path.resolve('./src/js/pages/') },
      { find: 'services', replacement: path.resolve('./src/js/services/') },
      { find: 'shared', replacement: path.resolve('./src/js/shared/') },
      { find: 'store', replacement: path.resolve('./src/js/store/') },
      { find: 'test', replacement: path.resolve('./src/js/test/') },
      { find: 'types', replacement: path.resolve('./src/js/types/') },
    ],
  },
  test: {
    globals: true,
    pool: 'vmThreads',
    environment: 'jsdom',
    environmentOptions: {
      jsdom: {
        // Pin to http://localhost (no port) to match the test environment expectation.
        url: 'http://localhost',
      },
    },
    setupFiles: [
      './src/js/test/setup-node-web-globals.js',
      './src/js/test/SetupTests.js',
    ],
    include: [
      'server/sys/castor/src/**/*.test.{ts,tsx}',
      'server/sys/castor/_mycrf/**/*.test.{ts,tsx}',
      'src/js/**/*.test.{ts,tsx}',
    ],
    exclude: ['src/js/apps/SurveyApp/**', '**/node_modules/**'],
    testTimeout: 20000,
    snapshotFormat: {
      escapeString: true,
      printBasicPrototype: true,
    },
    coverage: {
      provider: 'v8',
      reporter: ['lcov'],
      reportsDirectory: './build/jsCoverage',
      include: ['**/*.{js,jsx,ts,tsx}'],
      exclude: ['**/node_modules/**', '**/coverage/**', '**/server/**'],
    },
  },
});

System Info

macOS Tahoe; the CI runner is 8GB ubuntu 4 CPU

Used Package Manager

yarn

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• The provided reproduction is a minimal reproducible example of the bug.

[DucMinhNe]

The combination of symptoms — exit code 0 (Done in 397s is the package manager's success message), no summary stats, and truncated output — points to the vitest process dying (or being exited) mid-run rather than finishing and "forgetting" to fail. A completed run always prints the summary and exits non-zero on failures, on every pool. So I'd debug it as "what's killing the run early with code 0", and separately fix the speed problem that pushed you to vmThreads.

1. Check for a stray process.exit(0). This is the classic cause of exactly this triple symptom. With the default forks pool, a test (or dependency) calling process.exit kills only that child process and Vitest reports it as an error. Under vmThreads, tests share worker threads and the main process differently, so an exit can take down the whole run silently. Quick hunt:

// setupFiles
const realExit = process.exit;
process.exit = ((code?: number) => {
  console.trace(`process.exit(${code}) called`);
  return realExit(code as never);
}) as typeof process.exit;

2. Check for OOM worker crashes. vmThreads has a documented, known memory-leak problem: VM contexts aren't fully collectable, so long runs (your 25–31 min, 400+ files) steadily grow until a worker dies. A killed worker can also explain truncated output. Mitigation if you stay on vmThreads: set poolOptions.vmThreads.memoryLimit (recycles workers before they blow up). But honestly —

3. The better fix for both problems: pool: 'forks' with isolate: false. Most of vmThreads's speed win comes from not isolating per test file, not from VM magic. You can get the same win safely:

test: {
  pool: 'forks',
  isolate: false,   // big speedup; requires tests not to rely on per-file isolation
  // maxWorkers: tune to your CI runner
}

This keeps real process isolation (crashes are contained and reported, exit codes are reliable — which is exactly what your CI step needs) while typically landing in the same ballpark as vmThreads on wall-clock. If some test files genuinely need isolation, mark just those few with // @vitest-environment / per-project overrides instead of paying for isolation everywhere.

4. Belt-and-braces for CI: since you already emit junit, make the step's pass/fail come from the XML too (e.g. a junit-report action with fail_on_failure: true). That way even a hard process kill can't masquerade as success.

If you can grab the last ~50 lines of the vmThreads CI log (right before Done in …), it should show whether a worker died (OOM/tinypool error) vs. the process just exiting — happy to help interpret it.

[MILLERMARRU]

Worth double-checking the pool choice here, since forks (not threads) has been the documented default, and it's specifically the option that exists to avoid this class of symptom. The performance guide states forks is the default and is "generally preferred for avoiding issues like hanging processes or segfaults," with threads/vmThreads offered as an opt-in for raw speed at the cost of that safety margin.

vmThreads specifically carries this caveat straight from the config docs: "the VM module is unstable with ESM code and can lead to memory leaks... running code in a sandbox also means native module globals differ... ESM imports are cached indefinitely causing memory leaks with many contexts." This config pulls in @vitejs/plugin-react-swc, vite-plugin-svgr, and a CJS/ESM interop shim for @castoredc/i18n and @castoredc/matter, on an 8GB/4CPU CI runner. That combination lines up well with a worker silently OOMing or crashing partway through a 2500-case run, since vmThreads shares a real OS process (and its memory) across all VM contexts in that thread, unlike forks, where each worker is a separate process and a crash in one doesn't take the coordinator down with it.

A crashed thread worker not surfacing as a nonzero exit code, rather than a proper failure count, is consistent with the process dying mid-run before Vitest's own reporter gets to summarize results, exactly the "Done in 397.30s" with no stats and truncated output described here.

Two things worth trying instead of vmThreads:

pool: 'forks' with isolate: false and a tuned maxWorkers, keeping most of the speed win from disabling isolation without giving up process-level crash containment.

pool: 'vmForks' if the VM sandbox specifically is needed for something else, since it combines VM contexts with process-based workers rather than thread-based ones.

Also worth a quick sanity check independent of the pool: confirming the GHA step's actual exit code (echo $? right after the vitest invocation, before any output piping) isn't being swallowed by something else in that step, a | tee, a wrapping script, continue-on-error, since that produces the exact same silent-green-CI symptom regardless of pool.

[wbizmo]

I don't think the failing assertion itself is being ignored.

The combination of these three symptoms is the important part:

1. A failed test is visible in the output.
2. The normal Vitest summary is missing.
3. Yarn prints Done in 397.30s and the GitHub Actions step remains green.

A completed Vitest run should reach reporter finalization, print its test summary, write the final JUnit result, and return a non-zero status when a test fails. Since that final section is missing, I'd treat this as an interrupted run first and an exit-code bug second.

There are a few things I'd check to narrow this down.

1. Verify which process is actually returning 0

Done in ... is Yarn's final message, so I'd first bypass the package script and capture Vitest's exit code directly:

- name: Run Vitest directly
  shell: bash
  run: |
    set +e

    TZ=Europe/Amsterdam yarn exec vitest run \
      --pool=vmThreads \
      --reporter=default \
      --reporter=junit \
      --outputFile.junit=./build/junit.xml \
      --maxWorkers=2

    STATUS=$?

    echo "Vitest exit code: $STATUS"

    if [ -f ./build/junit.xml ]; then
      echo "JUnit generated"
      tail -n 20 ./build/junit.xml
    else
      echo "JUnit file missing"
    fi

    exit "$STATUS"

I'd also double-check the workflow isn't masking failures with:

continue-on-error: true

or something like:

vitest run || true

or

vitest run | tee test.log

without set -o pipefail.

---

2. Reduce it to a single failing test

Before blaming vmThreads, I'd try one deliberately failing test:

import { describe, expect, it } from 'vitest';

describe('exit-code reproduction', () => {
  it('fails intentionally', () => {
    expect(true).toBe(false);
  });
});

Run only that file:

yarn exec vitest run path/to/failing.test.ts \
  --pool=vmThreads \
  --reporter=default

echo $?

If that correctly exits with 1, then vmThreads itself can propagate failures normally and the issue is likely specific to the long-running suite.

---

3. Check for memory exhaustion

Your workload is exactly the type that stresses vmThreads:

• 400+ test files
• 2500+ test cases
• jsdom
• React SWC
• SVG transforms
• ESM/CJS interop
• 8 GB runner
• ~30 minute execution time

Vitest's documentation warns that VM pools can accumulate memory because ESM modules remain cached.

I'd enable heap logging:

test: {
  pool: 'vmThreads',
  vmMemoryLimit: '1GB',
  logHeapUsage: true,
  maxWorkers: 2,
}

If memory steadily increases until output suddenly stops, that would explain:

• missing summary
• truncated output
• no final reporter output

---

4. Search for explicit process termination

I'd also search for anything calling:

process.exit
process.kill
worker.terminate
parentPort.close

especially in setup files.

For debugging, I'd temporarily replace process.exit() with:

process.exit = ((code?: number) => {
  console.trace(`process.exit(${code ?? 0})`);
  throw new Error(`Unexpected process.exit(${code ?? 0})`);
}) as typeof process.exit;

That will immediately identify whether an application dependency or setup file is terminating the run before Vitest finishes.

---

My guess

Based on the missing summary rather than just the green CI, I'd rank the likely causes as:

1. The wrapper (Yarn or the GitHub Actions step) is swallowing Vitest's exit status.
2. A vmThreads worker is terminating before reporter finalization, likely due to memory pressure.
3. Something in the application or setup files is calling process.exit().
4. A genuine Vitest bug causing vmThreads to return 0 after a completed failing run.

The first thing I'd want to see is the direct exit code from yarn exec vitest run and whether the one-file failing reproduction returns 1 or 0. That would narrow this down significantly.

Let me know if this helps. If you can share the direct Vitest exit code, the last few lines of the CI log, and whether the JUnit XML is complete, I'd be happy to help dig into it further.