Skip to content

feat(agent-org): launch idempotent recoverable Follow-up Runs for #635 #645

Description

@ShiboSheng

Parent and dependencies

Goal

After explicit user confirmation, create exactly one new Agent Org Run linked to the selected historical source Run, prepare fresh Worker Sessions, and recover partial startup without reopening or mutating the source Run.

First-version policy: confirmation is always required. Automatic launch without user confirmation is not part of this issue.

User behavior

  • Confirming a proposal creates Run N · Starting and locks the Start action against duplicate clicks.
  • Successful materialization changes the same Run to Running and begins the confirmed goal.
  • A partial failure shows 启动未完成 for the same Run, performs a bounded automatic recovery, and exposes manual Retry and Cancel/Abandon actions.
  • Repeating confirmation or Retry never creates Run N+1 for the same accepted proposal.
  • If another Run is Starting, Running, or Paused, launch is refused with a readable conflict while the proposal remains available.

Required changes

  • Add Starting as a truthful non-terminal Run state and include it in the one-live-Run-per-Conversation invariant.
  • Implement idempotent org_follow_up_start using proposal identity plus an explicit request idempotency key.
  • In one SQLite writer transaction: validate pending proposal, terminal source Run, existing non-archived Conversation, and no live Run; snapshot the current Agent Org definition; create the new Run and lineage; map the shared Coordinator Session; persist durable per-member materialization intent and the confirmed goal; accept the proposal and record created_run_id; then commit.
  • Perform Worker Session creation and provider/Wake effects only after commit.
  • Create a fresh Worker Session for every member and add exact Run/Member/Session ownership as each idempotent materialization succeeds. Never reuse an old Worker.
  • Do not mark the Run Running or send initial work until every required member is durably ready.
  • Persist per-member materialization status and errors so app restart can resume the same Starting Run without duplicating external sessions.
  • Attempt at most three automatic recovery passes with capped backoff; after that retain a blocked recoverable Starting state and require manual Retry or Cancel/Abandon.
  • Treat post-commit Wake failure as a valid recoverable Run, not as permission to create another Run.
  • Update multi-Run deletion, Conversation archive/delete, nested-session checks, and cleanup so historical Runs and newer descendants are not accidentally removed or orphaned.

Invariants

  • One Conversation has at most one Starting, Running, or Paused Run.
  • Run N has fresh Tasks, Plans, Inbox, interventions, Recovery budgets, and Worker Sessions.
  • The source Run remains terminal and readable.
  • The shared Root Coordinator may map to sequential Runs; every Worker maps to exactly one Run.
  • All retry paths target the already-created Run and materialization records.
  • A post-commit acceleration failure preserves recovery evidence.

Acceptance criteria

  • Two concurrent confirmations return the same created Run.
  • Repeated network retries return the same Run and do not duplicate Workers.
  • Existing live Run conflict creates no partial Run or session rows.
  • Starting is visible and cannot be paused/resumed as if already Running.
  • Every new Worker Session maps only to the new Run.
  • An old Worker still resolves only to its historical Run.
  • Failure after some Workers are created resumes only missing members.
  • App restart during Starting resumes the same Run.
  • Automatic recovery is bounded to three passes; manual Retry continues the same Run.
  • Wake failure after commit does not encourage or permit duplicate launch.
  • Run 2 can execute, pause/resume, recover, and complete through existing production paths while Run 1 remains unchanged.
  • Deleting one historical Run does not delete a newer Run; deleting the Conversation follows an explicit full-cascade policy.

Verification

  • SQLite concurrency and idempotency tests with two writers.
  • Fault injection at every pre-commit and post-commit boundary.
  • Rust-native and CLI member materialization tests, including partial success and lost response cases.
  • Restart/recovery/watchdog tests with persisted generation and retry counters.
  • Multi-window confirmation and stale-result tests.
  • Existing Pause/Resume, Wake, Recovery, Approval, Finality, deletion, and launch suites.
  • Production-path Agent Org E2E proving fresh Worker ownership and no false success.
  • Scoped Clippy, formatting, architecture audit, and performance lifecycle matrix.

Out of scope

  • Automatic Follow-up launch without confirmation.
  • Concurrent live Runs in one Conversation.
  • Final Run selector and card visual implementation.

Estimated change size

  • Roughly 12–18 files.
  • Roughly 1,100–1,800 changed lines including tests.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions