Skip to content

Eval cleanup can stop a background server it did not start #71

Description

@benagentai93-dot

Baseline

Reproduced from v1.3.1 at 5a306f8956cb1eeae69f9709de0e4d61b44e11e7.

Reproduction

  1. Keep a live background server for a project, for example from agents-cli run --start-server.
  2. Run local eval generation in the same project.
  3. ensure_server() returns ServerInfo(..., started=False), but _run_against_local_server() unconditionally calls stop_server(project_root) in finally.

A deterministic monkeypatch regression supplies a reused ServerInfo and records the cleanup call; the reused server is stopped on both successful and exceptional eval paths. A related replacement race occurs because stop_server() rereads only the current PID file: if the invocation's server was replaced before finally, cleanup can target the replacement.

Actual behavior

Eval can stop a persistent server owned by another invocation. Cleanup identifies only the current project PID metadata, not the process this invocation actually started.

The run command has a sibling failure path: on a local transport error it forces cleanup even for a reused server or a server requested with --start-server.

Expected behavior

Automatic cleanup must run only for a process this invocation started, and only while the current PID metadata still identifies that process. Reused and user-persistent servers must survive success and failure. Explicit agents-cli run --stop-server should remain unconditional.

Minimal fix

Carry the owned PID from ensure_server(); use None for reuse. Pass that PID to a conditional stop_server(expected_pid=...) seam. Preserve the original --start-server cleanup decision across transport errors. No new dependency or lifecycle abstraction is needed.

Reference implementation: benagentai93-dot@b4646e6

Test evidence

Runnable regressions cover eval success/error with owned/reused servers, run success/error with and without --start-server, replacement PID refusal, and explicit stop behavior.

Fork verification:

  • focused lifecycle suite: 22 passed
  • full root suite: 41 passed
  • ruff check src tests: passed
  • ty check src: passed
  • uv build: passed
  • installed-wheel agents-cli --version: 1.3.1 on Python 3.11 and 3.13

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions