Skip to content

bug: benchmark script fails on Windows — execFileSync('npm', ...) needs .cmd + shell:true #966

@carlos-alm

Description

@carlos-alm

Found during dogfooding v3.9.4

Severity: Medium — blocks `npm run benchmark` on Windows.
Platform: Windows 11 x64, Node 22.18.0.
Command: `npm run benchmark -- --npm --version 3.9.4`

Reproduction

cd <codegraph-source-repo>
npm run benchmark -- --npm --version 3.9.4

Fails with:

Installing @optave/codegraph@3.9.4 into C:\Users\...\codegraph-bench-xxxx...
  Attempt 1 failed, retrying in 15s...
  ...
Error: Failed to install @optave/codegraph@3.9.4 after 5 attempts: spawnSync npm ENOENT
    at resolveBenchmarkSource (scripts/lib/bench-config.ts:82:11)

After a partial fix (switch to npm.cmd): spawnSync npm.cmd EINVAL.

Root cause

scripts/lib/bench-config.ts uses execFileSync('npm', [...], { ... }) three times (lines 74, 118, 149). On Windows:

  1. npm is actually npm.cmd in PATH — execFileSync('npm', ...) fails with ENOENT.
  2. Since Node 18.20 / 20.15, Node's security model refuses to spawnSync .cmd/.bat files without shell: true — fails with EINVAL.

See https://nodejs.org/api/child_process.html#spawning-bat-and-cmd-files-on-windows.

Suggested fix

Top of scripts/lib/bench-config.ts:

const NPM_CMD = os.platform() === 'win32' ? 'npm.cmd' : 'npm';
const NPM_SHELL = os.platform() === 'win32';

Replace all three execFileSync('npm', ...) calls with:

execFileSync(NPM_CMD, [...], {
  cwd: tmpDir,
  stdio: 'pipe',
  timeout: 120_000,
  shell: NPM_SHELL,
});

Verified working locally — with this patch, the benchmark completes both engine runs end-to-end (modulo the separate WASM crash tracked in the other issue).

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingdogfoodFound during dogfooding

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions