--test-concurrency has no effect on single test file

[hulkish]

Version

21.4.0

Platform

Darwin Stevens-MacBook-Pro.local 23.1.0 Darwin Kernel Version 23.1.0: Mon Oct 9 21:32:52 PDT 2023; root:xnu-10002.41.9~7/RELEASE_ARM64_T8122 arm64

Subsystem

No response

What steps will reproduce the bug?

1. create test file foo.test.mjs:

import { describe, it } from 'node:test';

// when ran with --test-concurrency of anything greater than 1, has no effect, uses concurrency of 1
describe('foo suite 1', () => {
  let i = 0;
  while (i++ < 50) {
    it(`test #${i}`, async () => {
      await new Promise((resolve) => setTimeout(resolve, 100));
    });
  }
});

// overrides any value of --test-concurrency with 2
describe('foo suite 2', {
  concurrency: 3,
}, () => {
  let i = 0;
  while (i++ < 100) {
    it(`test #${i}`, async () => {
      await new Promise((resolve) => setTimeout(resolve, 100));
    });
  }
});

// overrides any value of --test-concurrency with require('os').availableParallelism()
describe('foo suite 3', {
  concurrency: true,
}, () => {
  let i = 0;
  while (i++ < 10000) {
    it(`test #${i}`, async () => {
      await new Promise((resolve) => setTimeout(resolve, 100));
    });
  }
});

2. run:

# should default to require('os').availableParallelism(), but seems to default to concurrency of 1 or whatever value the test suite defines (aka { concurrecy: <value> })
node --test foo.test.mjs

# should run with concurrency of 7, but seems to still use 1 or whatever the test suite defines
node --test --test-concurrency=7 foo.test.mjs

# should run with concurrency of 1, but seems to still use 1 or whatever the test suite defines
node --test --test-concurrency=1 foo.test.mjs

How often does it reproduce? Is there a required condition?

Every time

What is the expected behavior? Why is that the expected behavior?

--test-concurrency should have an effect and behave as defined in the docs.

What do you see instead?

--test-concurrency has zero effect. You can see evidence of this by looking at the time each suite takes to complete:

# 50 tests
▶ foo suite 1 (5086.684417ms)
# 100 tests
▶ foo suite 2 (3457.998ms)
# 10000 tests
▶ foo suite 3 (658.40475ms)

Additional information

No response

[hulkish]

So, I've been playing with this a bit more.. and It seems that --test-concurrency and setting test('<name>', { concurrency: <value> }, () => {}); are two separate meanings of concurrency.

Per the test runner execution model as described in the docs:

Each matching test file is executed in a separate child process. The maximum number of child processes running at any time is controlled by the --test-concurrency flag. If the child process finishes with an exit code of 0, the test is considered passing. Otherwise, the test is considered to be a failure. Test files must be executable by Node.js, but are not required to use the node:test module internally.

Each test file is executed as if it was a regular script. That is, if the test file itself uses node:test to define tests, all of those tests will be executed within a single application thread, regardless of the value of the concurrency option of test().

So, I guess this means that --test-concurrency is exclusive to execution of multiple test files. (via child process) and { concurrency: <value> } in tests pertain essentially to async/promise based tests?