[Bug] Tests that spawn daemon subprocesses fail when run from inside an agent session (worker env and color flags leak via process.env) #1511
Replies: 2 comments
|
Fix details with exact locations (fork New helper — export function createRootDaemonEnv(overrides: NodeJS.ProcessEnv = {}): NodeJS.ProcessEnv {
const env: NodeJS.ProcessEnv = { ...process.env };
delete env[DAEMON_WORKER_ROLE_ENV];
delete env[DAEMON_WORKER_TOKEN_ENV];
delete env[DAEMON_WORKER_ACTIVE_SESSION_ID_ENV];
delete env[DAEMON_WORKER_RECOVERY_JOURNAL_ENV];
delete env[DAEMON_WORKER_SUPERVISOR_SOCKET_ENV];
delete env[ORPHAN_PROCESS_JOURNAL_ENV];
delete env[SESSION_LEASES_ENABLED_ENV];
delete env[SESSION_LEASE_OWNER_ID_ENV];
delete env.RLM_DEPTH;
return { ...env, ...overrides };
}Sanitize first, then apply overrides — so a test can still deliberately pass Applied at the three worker-env spawn sites:
Color flags —
Why this mirrors production rather than inventing a new list: the exact same variables are already stripped where production code spawns a root daemon —
The transport flip the tests were hitting is Net diff: 4 files changed, 45 insertions(+), 7 deletions(-), plus the 29-line helper. All previously failing tests pass when run from inside an agent session; clean-shell runners see no semantic change (the deleted variables are absent there). |
|
Confirming this from a different environment, and offering a containment measure that composes with your fix rather than replacing it. We run a soft fork of prime-agent and hit the same class on Linux (Ubuntu 24.04.4, Node 25.9) — so it is not macOS-specific. Our symptom was worse than failing tests: Your diagnosis explains the failures precisely, and the 1. Process reach is containable today, with no code change. An unprivileged PID namespace makes the host daemon's PID not exist for the test process, so a stray kill physically cannot land: SANDBOX=$(mktemp -d /tmp/prime-test-XXXX)
unshare --user --map-root-user --pid --fork --mount-proc \
env PRIME_AGENT_CODING_AGENT_DIR="$SANDBOX" \
npm --prefix packages/coding-agent run test:ciMeasured just now against a live daemon on pid 681507: This is defence in depth, not an alternative to sanitizing the spawn env: your fix makes the spawned daemon boot in the right mode, the namespace bounds the blast radius if any future test spawns something that signals broadly. 2. The second hazard is agent-dir writes, not just process reach. The suite never overrides On the side observation about both Happy to contribute the sandbox recipe as a docs note if that would be useful — following CONTRIBUTING.md, not opening anything unsolicited. |
Uh oh!
There was an error while loading. Please reload this page.
Affected area: Coding agent and CLI (test infrastructure)
What happened?
Running the test suite from inside a prime-agent session — i.e. asking an agent to run vitest, which is this repo's own documented workflow (
AGENTS.mdinstructs agents to run tests from the package root) — makes 20 tests fail deterministically across 4 files, in two distinct modes:test/daemon-supervisor-process.test.tsconnectEventuallyhandshake timeouttest/suite/regressions/4685-daemon-client-modes.test.tsstderrcleanliness assertions failtest/suite/regressions/4600-supervisor-singleton.test.tstest/suite/regressions/4606-update-restart-coordinator.test.tsTypical failure:
or:
Steps to reproduce
PRIME_AGENT_INTERNAL_DAEMON_WORKER=1,PRIME_AGENT_INTERNAL_DAEMON_WORKER_TOKEN,PRIME_AGENT_INTERNAL_DAEMON_SUPERVISOR_SOCKET,PRIME_AGENT_INTERNAL_DAEMON_WORKER_RECOVERY_JOURNAL,PRIME_AGENT_INTERNAL_ORPHAN_PROCESS_JOURNAL,SESSION_LEASES_*,RLM_DEPTH, and bothNO_COLOR=1andFORCE_COLOR=1).Minimal form: any
spawn(cli.ts, ["--mode", "daemon", ...], { env: { ...process.env } })from a worker-env parent reproduces it. CI and plain shells never see it because the parent env is clean there.Expected behavior
Tests that spawn a root daemon subprocess should boot it as a root daemon regardless of what env the test runner itself inherited; stderr-cleanliness assertions should not see warnings caused by the runner's own conflicting color flags.
Why this is a genuine deterministic failure, not flakiness or environment noise
PRIME_AGENT_INTERNAL_DAEMON_WORKER=1.AgentDaemon.handleConnection()pickstransport: "private-framed"when the daemon runs as a worker. Test helpers (spawnSupervisorindaemon-supervisor-process.test.ts,spawnRealSupervisorin4600-supervisor-singleton.test.ts,spawnSupervisorin4606-update-restart-coordinator.test.ts) spread...process.envverbatim, so the spawned "root" daemon silently boots in worker mode: it binds the socket and accepts connections, but every outbound frame is private-framed — the jsonlDaemonClientnever seesdaemon_helloand times out. A raw socket probe during reproduction confirmed it: a daemon spawned with the inherited env answersconnectwith{"kind":"outbound","outboundType":"daemon_hello","payloadEncoding":"jsonl"}wrapped in a binary private-frame header. Fails every run from a worker env; passes every run from a clean shell.v0.7.3(61131b2d1), no local changes —daemon-supervisor-process.test.tsfails identically; the other three files fail through the same env-inheritance path.daemon-mode.ts(supervisor respawn deletesDAEMON_WORKER_ROLE_ENV,DAEMON_WORKER_TOKEN_ENV,DAEMON_WORKER_ACTIVE_SESSION_ID_ENV,DAEMON_WORKER_RECOVERY_JOURNAL_ENV,DAEMON_WORKER_SUPERVISOR_SOCKET_ENV,ORPHAN_PROCESS_JOURNAL_ENV,SESSION_LEASES_ENABLED_ENV,SESSION_LEASE_OWNER_ID_ENV),daemon-launch.ts(~line 361), anddaemon-update-restart.ts(~line 521). fix(coding-agent): keep daemon root sessions at depth zero #1496 (e85a67ac4) addeddelete workerEnvironment.RLM_DEPTHtoDaemonSupervisorfor the same reason. The test helpers are the only CLI spawn sites without that sanitization.The
4685-daemon-client-modesfailures are a second, independent leak: agent sessions export bothNO_COLOR=1andFORCE_COLOR=1, so Node prints a warning to the stderr of every child process, breakingexpect(result.stderr).toBe("")style assertions. (Side observation: should the product set both flags at once? That may deserve its own fix — but tests spawning subprocesses shouldn't be sensitive to the runner's color config either way.)Prime Agent version: v0.7.3 (
61131b2d1), still present onmain(20b54977a)Environment: macOS 26.5.1; Node v22.23.2; vitest 4.1.10
Additional context
Fix implemented and verified on our fork (
ruttybob/prime-agent, branchdev-dogfood, commit8e8d26027):packages/coding-agent/test/daemon-spawn-env.tsexportingcreateRootDaemonEnv(overrides): copiesprocess.env, deletes the eight worker/lease variables plusRLM_DEPTH, then applies overrides — so tests that deliberately passRLM_DEPTH: "1"(the fix(coding-agent): keep daemon root sessions at depth zero #1496 regression test) still can.daemon-supervisor-process.test.ts,4600-supervisor-singleton.test.ts,4606-update-restart-coordinator.test.ts; color flags dropped in4685-daemon-client-modes.test.ts(runCli/runRpc).Happy to open a PR if the direction looks right.
All reactions