fix(scripts): spawn bench runners under process.execPath (#83921)#84304
fix(scripts): spawn bench runners under process.execPath (#83921)#84304hclsys wants to merge 1 commit into
Conversation
|
Thanks for the context here. I swept through the related work, and this is now duplicate or superseded. Close as superseded: the functional benchmark-spawner fix already landed on current main through the merged related PR, and this PR’s latest diff now only adds explanatory comments around code that is already fixed and covered by tests. Canonical path: Keep the merged code and test coverage from the superseding PR, and avoid merging a second comment-only branch unless maintainers explicitly want a separate comment-polish change. So I’m closing this here because the remaining work is already tracked in the canonical issue. Review detailsBest possible solution: Keep the merged code and test coverage from the superseding PR, and avoid merging a second comment-only branch unless maintainers explicitly want a separate comment-polish change. Do we have a high-confidence way to reproduce the issue? No against current main: both wrapper scripts already use Is this the best way to solve the issue? No, merging this PR is no longer the best way to solve the issue because the functional fix and test coverage already landed through the superseding PR. The current branch only adds explanatory comments. Security review: Security review cleared: The latest diff only adds source comments and introduces no dependencies, workflows, permissions, secret handling, package resolution, or code execution changes. What I checked:
Likely related people:
Codex review notes: model gpt-5.5, reasoning high; reviewed against 110042d840bb; fix evidence: commit 67c12e036802, main fix timestamp 2026-05-19T23:21:26-07:00. |
) `scripts/test-cli-startup-bench-budget.mjs` and `scripts/test-update-cli-startup-bench.mjs` both called `spawnSync("node", args, …)`, resolving the binary through PATH. In environments using nvm / fnm / volta / asdf the shim PATH-resolves to a different Node version than the one currently executing the parent script. The bench runner can then execute on Node X while the parent runs on Node Y, producing measurements that are not comparable to the baseline or to what CI uses; the fixture-update script writes a fixture that no other run can reproduce. `bench-cli-startup.ts` and `check-cli-startup-memory.mjs` already use `process.execPath` for exactly this reason. Bring the two test runners into line with the existing pattern.
3c961d3 to
bb1cd4d
Compare
|
Rebased onto current Validation after rebase:
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
|
ClawSweeper applied the proposed close for this PR.
|
Fixes #83921.
scripts/test-cli-startup-bench-budget.mjs:108andscripts/test-update-cli-startup-bench.mjs:67both invokedspawnSync("node", args, …). The first argument is resolved through PATH. In environments using version managers (nvm, fnm, volta, asdf) the PATH-resolvednodeis a shim that may resolve to a different Node version than the one currently executing the parent script. The bench runner can then run on Node X while the parent runs on Node Y — measurements stop being comparable to the baseline, and the fixture-updater writes a fixture that no other run can reproduce.The sibling scripts
scripts/bench-cli-startup.tsandscripts/check-cli-startup-memory.mjsalready useprocess.execPathfor exactly this reason. The inconsistency is what the issue calls out as subtle: the underlying bench harness usesprocess.execPathfor the CLI-under-test, but the runner / updater that drives it doesn't.Changes
scripts/test-cli-startup-bench-budget.mjs: changespawnSync("node", args, …)tospawnSync(process.execPath, args, …)inresolveCurrentReportPath's spawn block. Inline comment cites the sibling pattern inbench-cli-startup.ts/check-cli-startup-memory.mjsand the issue.scripts/test-update-cli-startup-bench.mjs: identical change in the top-level spawn at line 67.Diff stat: 2 files, +13 / -2. No
package.json, dependency, or interface change.Real behavior proof
Behavior or issue addressed: Sanitized issue evidence —
process.execPathis always the absolute path of the currently-executing Node binary, regardless of PATH state. The OpenClaw scripts dir already uses this pattern atbench-gateway-startup.ts:842,bench-gateway-restart.ts:1285,write-cli-startup-metadata.ts:208,246,openclaw-npm-postpublish-verify.ts:581, andbench-cli-startup.ts.Real environment tested: Local Node 22.x running under nvm (
/home/<user>/.nvm/versions/node/v22.22.0/bin/node) — the exact polyglot environment the issue's reproduction step calls out. Probe at/tmp/probe_83921.mjsdoes both halves of the proof. (a) Parses both patched scripts and verifies (i)spawnSync(process.execPath, argsappears in each, (ii) nospawnSync("node", argscalls remain. (b) Replays the structural difference between PATH-resolvednodeandprocess.execPathon the test machine — confirmsprocess.execPathis an absolute path (immune to PATH/shim drift), that runningspawnSync(process.execPath, ["--version"])returns a valid version string, and that the sibling scriptbench-cli-startup.tsalready uses the same pattern.Exact steps or command run after this patch:
node /tmp/probe_83921.mjsEvidence after fix:
Observed result after fix: Both bench runners now spawn the child Node process under the same binary as the parent, eliminating the version-mismatch class of bug entirely. No behavior change on systems where
node(PATH) andprocess.execPathalready agree (e.g., system Node, single-version Docker images).What was not tested: A live multi-version nvm switch demonstrating the buggy shape — the issue's repro requires nvm shell activation pointing to a different version than the parent script. The probe demonstrates the structural mechanism that makes the bug impossible (absolute-path-driven spawn rather than PATH lookup), which is the actual fix being made.
Audit (per CLAUDE rules — all 5 steps)
process.execPathis the codebase's canonical answer to this concern — already used in 5+ sibling scripts includingbench-cli-startup.ts,bench-gateway-startup.ts,bench-gateway-restart.ts,check-cli-startup-memory.mjs,write-cli-startup-metadata.ts. The fix is "use the existing primitive" not "add a helper." PASSmainflow. PASSgh pr list --search '83921 in:title,body'andgh pr list --search 'spawnSync node process.execPath'return no open PRs. Issue timeline shows zero cross-references. PASSgit log --oneline -3 -- scripts/test-cli-startup-bench-budget.mjs scripts/test-update-cli-startup-bench.mjsshowse1061a8b46 test(live): tolerate provider drift in release checks— unrelated. PASSspawnSync.