Two related issues in Detection/main_benchmark.py, TaskExecutor:
_execute_command runs the claude CLI via asyncio.create_subprocess_exec + asyncio.wait_for(process.communicate(), timeout=...). wait_for only cancels the await — on timeout, the child process (and any MCP server subprocesses it spawned) is left running, unkilled. guardrail/adr_agent/adr_baseline.py invokes the same CLI via subprocess.run(..., timeout=...), which does kill the child on timeout — the correct pattern already exists elsewhere in the repo.
execute_ads_task computes existing_sessions via a full recursive rglob("*.jsonl") over ~/.claude/projects/ (host-wide, unrelated to the benchmark, grows with unrelated usage) and passes it to _process_results, which never references the parameter — confirmed by grep. This scan runs, and its result is discarded, once per benchmark task.
At benchmark scale (hundreds of tasks, README cites 303 tasks / 133 MCP servers), every timeout leaks a process tree, and the dead scan adds wasted I/O to every task regardless of timeout.
I already have a fix scoped: terminate the process group on timeout, mirroring the safe pattern adr_baseline.py already uses elsewhere in this repo, and remove the unused existing_sessions computation. Opening this first for visibility — PR to follow.
Two related issues in
Detection/main_benchmark.py,TaskExecutor:_execute_commandruns theclaudeCLI viaasyncio.create_subprocess_exec+asyncio.wait_for(process.communicate(), timeout=...).wait_foronly cancels the await — on timeout, the child process (and any MCP server subprocesses it spawned) is left running, unkilled.guardrail/adr_agent/adr_baseline.pyinvokes the same CLI viasubprocess.run(..., timeout=...), which does kill the child on timeout — the correct pattern already exists elsewhere in the repo.execute_ads_taskcomputesexisting_sessionsvia a full recursiverglob("*.jsonl")over~/.claude/projects/(host-wide, unrelated to the benchmark, grows with unrelated usage) and passes it to_process_results, which never references the parameter — confirmed by grep. This scan runs, and its result is discarded, once per benchmark task.At benchmark scale (hundreds of tasks, README cites 303 tasks / 133 MCP servers), every timeout leaks a process tree, and the dead scan adds wasted I/O to every task regardless of timeout.
I already have a fix scoped: terminate the process group on timeout, mirroring the safe pattern
adr_baseline.pyalready uses elsewhere in this repo, and remove the unusedexisting_sessionscomputation. Opening this first for visibility — PR to follow.