Skip to content

Timed-out claude CLI subprocess is never killed in benchmark task execution; dead code does a redundant unbounded directory scan #18

Description

@Rahul-s-007

Two related issues in Detection/main_benchmark.py, TaskExecutor:

  1. _execute_command runs the claude CLI via asyncio.create_subprocess_exec + asyncio.wait_for(process.communicate(), timeout=...). wait_for only cancels the await — on timeout, the child process (and any MCP server subprocesses it spawned) is left running, unkilled. guardrail/adr_agent/adr_baseline.py invokes the same CLI via subprocess.run(..., timeout=...), which does kill the child on timeout — the correct pattern already exists elsewhere in the repo.
  2. execute_ads_task computes existing_sessions via a full recursive rglob("*.jsonl") over ~/.claude/projects/ (host-wide, unrelated to the benchmark, grows with unrelated usage) and passes it to _process_results, which never references the parameter — confirmed by grep. This scan runs, and its result is discarded, once per benchmark task.

At benchmark scale (hundreds of tasks, README cites 303 tasks / 133 MCP servers), every timeout leaks a process tree, and the dead scan adds wasted I/O to every task regardless of timeout.

I already have a fix scoped: terminate the process group on timeout, mirroring the safe pattern adr_baseline.py already uses elsewhere in this repo, and remove the unused existing_sessions computation. Opening this first for visibility — PR to follow.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions