Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add --no-file-parallelism, --maxWorkers, --minWorkers flags #4705

Merged
merged 6 commits into from
Dec 19, 2023
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
26 changes: 26 additions & 0 deletions docs/config/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -879,6 +879,32 @@ Pass additional arguments to `node` process in the VM context. See [Command-line
Be careful when using, it as some options may crash worker, e.g. --prof, --title. See https://github.com/nodejs/node/issues/41103.
:::

### parallelism

- **Type:** `boolean`
- **Default:** `true`
- **Version:** Since Vitest 1.1

Should all test files run in parallel. This will override `maxWorkers` and `minWorkers` options.
sheremet-va marked this conversation as resolved.
Show resolved Hide resolved

::: tip
This option doesn't affect tests running in the same file. If you want to run those in parallel, use `concurrent` option on [describe](/api/#describe-concurrent) or via [a config](#sequence-concurrent).
:::

### maxWorkers

- **Type:** `number`
- **Version:** Since Vitest 1.1

Maximum number of workers to run tests in. `poolOptions.{threads,vmThreads}.maxThreads`/`poolOptions.forks.maxForks` has higher priority.

### minWorkers

- **Type:** `number`
- **Version:** Since Vitest 1.1

Minimum number of workers to run tests in. `poolOptions.{threads,vmThreads}.minThreads`/`poolOptions.forks.minForks` has higher priority.

### testTimeout

- **Type:** `number`
Expand Down
3 changes: 3 additions & 0 deletions docs/guide/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,9 @@ Run only [benchmark](https://vitest.dev/guide/features.html#benchmarking-experim
| `--poolOptions <options>` | Specify pool options |
| `--poolOptions.threads.isolate` | Isolate tests in threads pool (default: `true`) |
| `--poolOptions.forks.isolate` | Isolate tests in forks pool (default: `true`) |
| `--parallelism` | Should all test files run in parallel. Use --no-parallelism to disable (default: true) |
| `--maxWorkers` | Maximum number of workers to run tests in |
| `--minWorkers` | Minimum number of workers to run tests in |
| `--silent` | Silent console output from tests |
| `--reporter <name>` | Select reporter: `default`, `verbose`, `dot`, `junit`, `json`, or a path to a custom reporter |
| `--outputFile <filename/-s>` | Write test results to a file when the `--reporter=json` or `--reporter=junit` option is also specified <br /> Via [cac's dot notation] you can specify individual outputs for multiple reporters |
Expand Down
7 changes: 7 additions & 0 deletions docs/guide/debugging.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,13 @@ vitest --inspect-brk --pool threads --poolOptions.threads.singleThread
vitest --inspect-brk --pool forks --poolOptions.forks.singleFork
```

If you are using Vitest 1.1 or higher, you can also just provide `--no-parallelism` flag:

```sh
# If pool is unknown
vitest --inspect-brk --no-parallelism
```

Once Vitest starts it will stop execution and wait for you to open developer tools that can connect to [Node.js inspector](https://nodejs.org/en/docs/guides/debugging-getting-started/). You can use Chrome DevTools for this by opening `chrome://inspect` on browser.

In watch mode you can keep the debugger open during test re-runs by using the `--poolOptions.threads.isolate false` options.
3 changes: 3 additions & 0 deletions packages/vitest/src/node/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,9 @@ cli
.option('--poolOptions <options>', 'Specify pool options')
.option('--poolOptions.threads.isolate', 'Isolate tests in threads pool (default: true)')
.option('--poolOptions.forks.isolate', 'Isolate tests in forks pool (default: true)')
.option('--parallelism', 'Should all test files run in parallel. Use --no-parallelism to disable (default: true)')
.option('--maxWorkers', 'Maximum number of workers to run tests in')
.option('--minWorkers', 'Minimum number of workers to run tests in')
.option('--environment <env>', 'Specify runner environment, if not running in the browser (default: node)')
.option('--passWithNoTests', 'Pass when no tests found')
.option('--logHeapUsage', 'Show the size of heap for each test')
Expand Down
12 changes: 10 additions & 2 deletions packages/vitest/src/node/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -113,13 +113,21 @@ export function resolveConfig(
resolved.shard = { index, count }
}

resolved.parallelism ??= true

if (!resolved.parallelism) {
// ignore user config, parallelism cannot be implemented without limiting workers
resolved.maxWorkers = 1
resolved.minWorkers = 1
}

if (resolved.inspect || resolved.inspectBrk) {
const isSingleThread = resolved.pool === 'threads' && resolved.poolOptions?.threads?.singleThread
const isSingleFork = resolved.pool === 'forks' && resolved.poolOptions?.forks?.singleFork

if (!isSingleThread && !isSingleFork) {
if (resolved.parallelism && !isSingleThread && !isSingleFork) {
const inspectOption = `--inspect${resolved.inspectBrk ? '-brk' : ''}`
throw new Error(`You cannot use ${inspectOption} without "poolOptions.threads.singleThread" or "poolOptions.forks.singleFork"`)
throw new Error(`You cannot use ${inspectOption} without "--no-parallelism", "poolOptions.threads.singleThread" or "poolOptions.forks.singleFork"`)
}
}

Expand Down
20 changes: 11 additions & 9 deletions packages/vitest/src/node/pools/child.ts
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,10 @@ export function createChildProcessPool(ctx: Vitest, { execArgv, env, forksPath }
? Math.max(Math.floor(numCpus / 2), 1)
: Math.max(numCpus - 1, 1)

const maxThreads = ctx.config.poolOptions?.forks?.maxForks ?? threadsCount
const minThreads = ctx.config.poolOptions?.forks?.minForks ?? threadsCount
const poolOptions = ctx.config.poolOptions?.forks ?? {}

const maxThreads = poolOptions.maxForks ?? ctx.config.maxWorkers ?? threadsCount
const minThreads = poolOptions.minForks ?? ctx.config.minWorkers ?? threadsCount

const options: TinypoolOptions = {
runtime: 'child_process',
Expand All @@ -70,20 +72,20 @@ export function createChildProcessPool(ctx: Vitest, { execArgv, env, forksPath }

env,
execArgv: [
...ctx.config.poolOptions?.forks?.execArgv ?? [],
...poolOptions.execArgv ?? [],
...execArgv,
],

terminateTimeout: ctx.config.teardownTimeout,
concurrentTasksPerWorker: 1,
sheremet-va marked this conversation as resolved.
Show resolved Hide resolved
}

if (ctx.config.poolOptions?.forks?.isolate ?? true) {
const isolated = poolOptions.isolate ?? true

if (isolated)
options.isolateWorkers = true
options.concurrentTasksPerWorker = 1
}

if (ctx.config.poolOptions?.forks?.singleFork) {
options.concurrentTasksPerWorker = 1
if (poolOptions.singleFork || !ctx.config.parallelism) {
options.maxThreads = 1
options.minThreads = 1
}
Expand Down Expand Up @@ -164,7 +166,7 @@ export function createChildProcessPool(ctx: Vitest, { execArgv, env, forksPath }
const files = Object.values(filesByEnv).flat()
const results: PromiseSettledResult<void>[] = []

if (ctx.config.poolOptions?.forks?.isolate ?? true) {
if (isolated) {
results.push(...await Promise.allSettled(files.map(({ file, environment, project }) =>
runFiles(project, getConfig(project), [file], environment, invalidates))))
}
Expand Down
22 changes: 12 additions & 10 deletions packages/vitest/src/node/pools/threads.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,34 +43,36 @@ export function createThreadsPool(ctx: Vitest, { execArgv, env, workerPath }: Po
? Math.max(Math.floor(numCpus / 2), 1)
: Math.max(numCpus - 1, 1)

const maxThreads = ctx.config.poolOptions?.threads?.maxThreads ?? threadsCount
const minThreads = ctx.config.poolOptions?.threads?.minThreads ?? threadsCount
const poolOptions = ctx.config.poolOptions?.threads ?? {}

const maxThreads = poolOptions.maxThreads ?? ctx.config.maxWorkers ?? threadsCount
const minThreads = poolOptions.minThreads ?? ctx.config.minWorkers ?? threadsCount

const options: TinypoolOptions = {
filename: workerPath,
// TODO: investigate further
// It seems atomics introduced V8 Fatal Error https://github.com/vitest-dev/vitest/issues/1191
useAtomics: ctx.config.poolOptions?.threads?.useAtomics ?? false,
useAtomics: poolOptions.useAtomics ?? false,

maxThreads,
minThreads,

env,
execArgv: [
...ctx.config.poolOptions?.threads?.execArgv ?? [],
...poolOptions.execArgv ?? [],
...execArgv,
],

terminateTimeout: ctx.config.teardownTimeout,
concurrentTasksPerWorker: 1,
}

if (ctx.config.poolOptions?.threads?.isolate ?? true) {
const isolated = poolOptions.isolate ?? true

if (isolated)
options.isolateWorkers = true
options.concurrentTasksPerWorker = 1
}

if (ctx.config.poolOptions?.threads?.singleThread) {
options.concurrentTasksPerWorker = 1
if (poolOptions.singleThread || !ctx.config.parallelism) {
options.maxThreads = 1
options.minThreads = 1
}
Expand Down Expand Up @@ -144,7 +146,7 @@ export function createThreadsPool(ctx: Vitest, { execArgv, env, workerPath }: Po
const files = Object.values(filesByEnv).flat()
const results: PromiseSettledResult<void>[] = []

if (ctx.config.poolOptions?.threads?.isolate ?? true) {
if (isolated) {
results.push(...await Promise.allSettled(files.map(({ file, environment, project }) =>
runFiles(project, getConfig(project), [file], environment, invalidates))))
}
Expand Down
2 changes: 1 addition & 1 deletion packages/vitest/src/node/pools/typecheck.ts
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ export function createTypecheckPool(ctx: Vitest): ProcessPool {
setTimeout(() => {
resolve(false)
clearInterval(_i)
}, 500)
}, 500).unref()
})
const triggered = await _p
if (project.typechecker && !triggered) {
Expand Down
14 changes: 8 additions & 6 deletions packages/vitest/src/node/pools/vm-threads.ts
Original file line number Diff line number Diff line change
Expand Up @@ -48,14 +48,16 @@ export function createVmThreadsPool(ctx: Vitest, { execArgv, env, vmPath }: Pool
? Math.max(Math.floor(numCpus / 2), 1)
: Math.max(numCpus - 1, 1)

const maxThreads = ctx.config.poolOptions?.vmThreads?.maxThreads ?? threadsCount
const minThreads = ctx.config.poolOptions?.vmThreads?.minThreads ?? threadsCount
const poolOptions = ctx.config.poolOptions?.vmThreads ?? {}

const maxThreads = poolOptions.maxThreads ?? ctx.config.maxWorkers ?? threadsCount
const minThreads = poolOptions.minThreads ?? ctx.config.minWorkers ?? threadsCount

const options: TinypoolOptions = {
filename: vmPath,
// TODO: investigate further
// It seems atomics introduced V8 Fatal Error https://github.com/vitest-dev/vitest/issues/1191
useAtomics: ctx.config.poolOptions?.vmThreads?.useAtomics ?? false,
useAtomics: poolOptions.useAtomics ?? false,

maxThreads,
minThreads,
Expand All @@ -66,16 +68,16 @@ export function createVmThreadsPool(ctx: Vitest, { execArgv, env, vmPath }: Pool
'--experimental-vm-modules',
'--require',
suppressWarningsPath,
...ctx.config.poolOptions?.vmThreads?.execArgv ?? [],
...poolOptions.execArgv ?? [],
...execArgv,
],

terminateTimeout: ctx.config.teardownTimeout,
concurrentTasksPerWorker: 1,
maxMemoryLimitBeforeRecycle: getMemoryLimit(ctx.config) || undefined,
}

if (ctx.config.poolOptions?.vmThreads?.singleThread) {
options.concurrentTasksPerWorker = 1
if (poolOptions.singleThread || !ctx.config.parallelism) {
options.maxThreads = 1
options.minThreads = 1
}
Expand Down
17 changes: 17 additions & 0 deletions packages/vitest/src/types/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -294,6 +294,23 @@ export interface InlineConfig {
*/
poolOptions?: PoolOptions

/**
* Maximum number of workers to run tests in. `poolOptions.{threads,vmThreads}.maxThreads`/`poolOptions.forks.maxForks` has higher priority.
*/
maxWorkers?: number
/**
* Minimum number of workers to run tests in. `poolOptions.{threads,vmThreads}.minThreads`/`poolOptions.forks.minForks` has higher priority.
*/
minWorkers?: number

/**
* Should all test files run in parallel. Doesn't affect tests running in the same file.
* This will override `maxWorkers` and `minWorkers` options.
sheremet-va marked this conversation as resolved.
Show resolved Hide resolved
*
* @default true
*/
parallelism?: boolean

/**
* Automatically assign pool based on globs. The first match will be used.
*
Expand Down
6 changes: 3 additions & 3 deletions test/config/test/failures.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,19 +33,19 @@ test('shard index must be smaller than count', async () => {
test('inspect requires changing pool and singleThread/singleFork', async () => {
const { stderr } = await runVitest({ inspect: true })

expect(stderr).toMatch('Error: You cannot use --inspect without "poolOptions.threads.singleThread" or "poolOptions.forks.singleFork"')
expect(stderr).toMatch('Error: You cannot use --inspect without "--no-parallelism", "poolOptions.threads.singleThread" or "poolOptions.forks.singleFork"')
})

test('inspect cannot be used with multi-threading', async () => {
const { stderr } = await runVitest({ inspect: true, pool: 'threads', poolOptions: { threads: { singleThread: false } } })

expect(stderr).toMatch('Error: You cannot use --inspect without "poolOptions.threads.singleThread" or "poolOptions.forks.singleFork"')
expect(stderr).toMatch('Error: You cannot use --inspect without "--no-parallelism", "poolOptions.threads.singleThread" or "poolOptions.forks.singleFork"')
})

test('inspect-brk cannot be used with multi processing', async () => {
const { stderr } = await runVitest({ inspect: true, pool: 'forks', poolOptions: { forks: { singleFork: false } } })

expect(stderr).toMatch('Error: You cannot use --inspect without "poolOptions.threads.singleThread" or "poolOptions.forks.singleFork"')
expect(stderr).toMatch('Error: You cannot use --inspect without "--no-parallelism", "poolOptions.threads.singleThread" or "poolOptions.forks.singleFork"')
})

test('c8 coverage provider is not supported', async () => {
Expand Down