epic(cli): headless exit-code conventions for runs and goals

[dennisonbertram]

Parent: #803

Objective

Give go-code's headless CLI a documented, tested exit-code contract so shell scripts and CI can branch on run outcomes. Today harnesscli -prompt ... exits 0 for every terminal run state — including run.failed and run.cancelled — so automation cannot distinguish success from failure without parsing stdout. This matches kimi-code's kimi -p behavior, which exits 0 when a goal completes, 3 when it blocks, 6 when it pauses, and non-zero on turn failure.

Context

The one-shot streaming path already returns int exit codes end to end: main() calls exitFunc(runCommand(osArgs[1:])) (cmd/harnesscli/main.go:136-138, exitFunc = os.Exit at :30), and run(args []string) int (cmd/harnesscli/main.go:140) returns 1 on usage/transport errors and 130 on SIGINT/SIGTERM via handleStreamError (cmd/harnesscli/main.go:228). But after streamRunEvents delivers a terminal event, run prints terminal_event=<type> and returns 0 unconditionally (cmd/harnesscli/main.go:219-220), and runContinue does the same (cmd/harnesscli/runctl.go:324-330). The harness already emits typed terminal events — run.completed, run.failed, run.cancelled (internal/harness/events.go:20-34, IsTerminalEvent at :472) — and non-terminal waiting states (run.waiting_for_user at internal/harness/events.go:22; RunStatusWaitingForUser / RunStatusWaitingForApproval at internal/harness/types.go:337-338) that a headless caller can interpret as "blocked". Goal statuses exist in internal/goals/goals.go:25-30 (pending/running/verifying/completed/failed/cancelled) but are only exposed through the in-run goals tool (internal/harness/tools_default.go:439-440); there is no /v1/goals route and no goal CLI subcommand, and no blocked/paused goal status. kimi-code's contract makes headless usage scriptable; go-code needs the same guarantee documented and enforced by tests.

The go-code wrapper script needs no changes: in scripts/go-code.sh the harnesscli invocation is the last command of main for both prompt and cli modes (scripts/go-code.sh:393, :399), and main "$@" is the script's final line (:404), so the harnesscli exit status already propagates as the wrapper's own exit status (the stop_server EXIT trap does not override it). Slice 4 adds an explicit assertion of this propagation so a future wrapper refactor cannot silently break it.

Existing Foundation (reuse — do not reimplement)

• cmd/harnesscli/main.go:140 — run(args []string) int; all one-shot outcomes already funnel through int return codes and the exitFunc indirection (testable without subprocess).
• cmd/harnesscli/main.go:228 — handleStreamError; establishes the 130-on-interrupt convention to preserve.
• internal/harness/events.go:472 — IsTerminalEvent; canonical terminal-event set (run.completed/run.failed/run.cancelled, constants at :20-34).
• internal/harness/events.go:22 and internal/harness/types.go:337-338 — run.waiting_for_user event and waiting run statuses used for blocked detection.
• internal/harness/events.go:384 — EventMaxTurnsExhausted; non-terminal signal the contract must classify (folds into run failure semantics).
• cmd/harnesscli/runctl.go:260 — runContinue; second streaming entry point that must honor the same mapping.
• cmd/harnesscli/auth.go:101 — dispatch; confirms which subcommands stream (continue) vs. exit immediately (cancel, status, list).
• cmd/harnesscli/askuser.go:52 — handleAskUserQuestion; stdin/stdout answer flow for run.waiting_for_user, currently not wired into the streamRunEvents loop (call sites are tests only) — relevant to defining "blocked" in non-interactive mode.
• internal/goals/goals.go:25-30 — goal Status values the contract reserves codes for.
• internal/fakeprovider/provider.go — scripted Turn provider with ExhaustedBehavior (incl. ExhaustError) for deterministic terminal-outcome tests.
• test/e2e/e2e_test.go + test/e2e/helpers_test.go — real HTTP+SSE e2e harness patterns (start run, drain until terminal, cancel mid-run).
• cmd/harnesscli/main_test.go:17-42 — runCommand/exitFunc test-double pattern for asserting exit codes in-process.

Scope

• A single documented exit-code contract for headless harnesscli usage (one-shot -prompt and continue with streaming), adopting kimi-compatible codes where semantics align. Proposed table for Slice 1 to ratify:
  • 0 — terminal run.completed (success)
  • 1 — client-side errors: bad flags, missing prompt, connection/HTTP failures (current behavior, unchanged)
  • 2 — terminal run.failed (turn failure; satisfies kimi's "non-zero on turn failure")
  • 3 — blocked: the run cannot proceed without input it will never get headlessly — run.waiting_for_user, tool.approval_required, or plan.approval_required observed while stdin is non-interactive (kimi-compatible); reserved for goal-blocked semantics
  • 6 — paused/cancelled: terminal run.cancelled; work is interrupted but resumable via harnesscli continue (kimi-compatible)
  • 130 — SIGINT/SIGTERM (current handleStreamError behavior, unchanged)
• Implement the mapping in the one-shot path (run) and continue streaming path (runContinue).
• Preserve the existing stdout contract (run_id=<id> and terminal_event=<event_type> lines); only the process exit code changes, so existing log parsers keep working.
• Blocked detection in one-shot mode: when a waiting event arrives and stdin is not a terminal, report the blocked state and exit 3 instead of streaming forever.
• e2e coverage asserting the exit code for each terminal outcome (including propagation through the go-code wrapper), plus CLI reference documentation.

Out of Scope

• TUI exit behavior (--tui); the contract covers headless/streaming mode only.
• Interactive ask-user wiring into the one-shot stream loop (handleAskUserQuestion integration is a separate epic's concern).
• A goal-scoped headless entrypoint (no /v1/goals route or goal subcommand exists today); the contract only reserves 3/6 for future goal blocked/paused semantics and documents the mapping for internal/goals statuses.
• Changing server-side run/goal state machines, event types, or SSE framing.
• Non-streaming subcommands (list, status, cancel, replay, search) keep their current 0/1 codes; the contract documents this but does not change them.

Proposed PR Slices

Slices are sequential, not parallel-safe: Slice 2 implements the contract Slice 1 ratifies, Slice 3 extends Slice 2's mapping into the stream loop, and Slice 4's e2e assertions cover the behavior of Slices 2 and 3.

Slice 1: docs(cli): specify headless exit-code contract for runs and goals

Goal: Ratify and document the exit-code table before any behavior changes, so implementation slices have a fixed target.
Key changes:

• New website/docs/reference/exit-codes.md: the full mapping table (run terminal events, waiting states, client errors, interrupt), the kimi-code 0/3/6 alignment rationale, and the reserved goal-status mapping (completed/failed/cancelled today; blocked/paused reserved).
• Cross-link from website/docs/cli/harnesscli.md (which today only documents "Print terminal_event=<event_type> and exit", :41) and website/docs/reference/events-catalog.md.
• Explicitly record that max_turns.exhausted (internal/harness/events.go:384) is non-terminal and surfaces through the subsequent terminal event's code.
• Explicitly record the cost-limit case: run.cost_limit_reached (internal/harness/events.go:27) terminates the run with run.completed (not run.failed), so it exits 0 — the contract should call this out so scripting users are not surprised.
• Record the blocked signals precisely: run.waiting_for_user (:22) for question-blocked runs, and tool.approval_required / plan.approval_required (:69, :83) for approval-blocked runs (status waiting_for_approval, internal/harness/types.go:338; there is no dedicated waiting-for-approval event).
• Include a "current vs. contracted behavior" table so reviewers can see exactly which process outcomes change (failed 0→2, cancelled 0→6, blocked infinite-stream→3) and which stay fixed (success 0, client error 1, interrupt 130).
  Tests: Docs-only slice; no behavior tests apply. Reviewer validates the table against the event constants in internal/harness/events.go.
  Acceptance: Contract page merged; every code in the table traces to an existing event constant, run status, or current CLI behavior.

Slice 2: feat(cli): map run terminal events to exit codes in one-shot and continue

Goal: Failed and cancelled runs stop exiting 0.
Key changes:

• New cmd/harnesscli/exitcodes.go: package-level constants (exitSuccess, exitClientError, exitRunFailed, exitBlocked, exitCancelled, exitInterrupted) and exitCodeForTerminalEvent(harness.EventType) int implementing the Slice 1 table; unknown/empty event types map to 1 (defensive default, non-zero so a scripting caller never mistakes them for success).
• cmd/harnesscli/main.go run(): return exitCodeForTerminalEvent(...) instead of unconditional 0 at :219-220; keep the terminal_event= stdout line and the 1/130 paths untouched.
• cmd/harnesscli/runctl.go runContinue(): same mapping at :324-330 (-no-stream stays 0/1; it never observes a terminal event).
• Replace the literal 1/130 returns in run and handleStreamError with the named constants so the contract has exactly one source of truth.
  Tests: Unit tests for exitCodeForTerminalEvent covering every terminal event plus unknown events; CLI-level tests using the existing httptest.NewServer SSE pattern in cmd/harnesscli/main_test.go asserting run() returns 0/2/6 for completed/failed/cancelled streams.
  Acceptance: go test ./cmd/harnesscli/... green; a scripted server emitting run.failed makes the one-shot path exit 2, run.cancelled exits 6, run.completed exits 0.

Slice 3: feat(cli): exit 3 when a headless run blocks on user input

Goal: A non-interactive caller gets a deterministic blocked signal instead of an infinite stream.
Key changes:

• cmd/harnesscli/main.go stream loop: processSSEBlock currently only signals terminal events back to streamRunEvents; extend that path so a run.waiting_for_user, tool.approval_required, or plan.approval_required event is surfaced to the caller as a blocked notification.
• When stdin is not a terminal (term.IsTerminal, same check runTUI already uses at cmd/harnesscli/main.go:517), the one-shot path prints the blocked reason and run ID to stderr, stops streaming, and returns 3. The server-side run is left intact (resumable via harnesscli continue); no auto-cancel.
• Keep interactive-stdin behavior unchanged (stream stays open; no new prompt UI — that is the ask-user epic's scope). Note handleAskUserQuestion (cmd/harnesscli/askuser.go:52) exists for that future wiring and is not reused here.
  Tests: behavior tests in cmd/harnesscli driving run() against an httptest server that emits run.waiting_for_user (and a second case emitting tool.approval_required) then holds the stream, asserting exit 3, the stderr message, and that no cancel POST reaches the server; plus a TTY-stdin case (or injected terminal-detection double) asserting the stream is not aborted.
  Acceptance: With piped stdin, a run that emits run.waiting_for_user exits 3 with a stderr message naming the run ID and the continue command; with a TTY stdin, behavior is unchanged.

Slice 4: test(e2e): end-to-end exit-code assertions per terminal outcome

Goal: Prove the contract over real HTTP+SSE against a real server, and publish the user-facing reference.
Key changes:

• New e2e coverage in test/e2e/ (following e2e_test.go/helpers_test.go patterns) driving the CLI's run() against a test server backed by scripted providers (internal/fakeprovider Turn scripts for completion; provider error or ExhaustError for failure; blocking provider + POST /v1/runs/{id}/cancel for cancelled), asserting exit codes 0, 2, and 6 respectively.
• One wrapper-level test or scripted check (extending cmd/harnesscli/go_code_script_test.go if that is where wrapper behavior is pinned — confirm during implementation) asserting go-code "prompt" surfaces the harnesscli exit code unchanged.
• Update website/docs/cli/harnesscli.md and website/docs/reference/cli-flags.md to state each headless command's exit codes and link the contract page.
• Add the new e2e file(s) to the package's existing patterns so they run under the standard go test ./test/e2e/... invocation used by the repo's regression flow (scripts/test-regression.sh).
  Tests: The e2e assertions are the tests; full go test ./test/e2e/... and go test ./cmd/harnesscli/... green.
  Acceptance: CI runs e2e assertions for all three terminal outcomes plus wrapper propagation; docs show the same table as website/docs/reference/exit-codes.md; a manual smoke (harnesscli -prompt ... ; echo $? against a local harnessd with a failing provider) matches the documented codes.

Acceptance Criteria

• harnesscli -prompt ... exits 0 on run.completed, 2 on run.failed, 6 on run.cancelled, 3 on blocked-with-non-interactive-stdin, 130 on SIGINT/SIGTERM, 1 on client/transport errors — each covered by an automated test.
• harnesscli continue <run-id> <prompt> (streaming) applies the same terminal-event mapping.
• The blocked path covers both question-blocked (run.waiting_for_user) and approval-blocked (tool.approval_required / plan.approval_required) runs, and never cancels the server-side run.
• stdout output is unchanged (run_id= / terminal_event= lines preserved); only the process exit code changes.
• The contract is documented in website/docs/reference/exit-codes.md and cross-linked from the CLI reference; documented codes match the code exactly.
• Every slice developed in a worktree created via scripts/init.sh, TDD per docs/runbooks/testing.md (tests written first, no trivial tests), merged via scripts/verify-and-merge.sh with all tests green.
• No regressions: go test ./cmd/harnesscli/... ./internal/harness/... ./test/e2e/... green on every slice.

Test Requirements

• Strict TDD per docs/runbooks/testing.md: behavior tests land before implementation in each code slice.
• Unit tests for the event→code mapping must cover all three terminal events, waiting events, unknown/empty event types, and the unchanged 1/130 paths — using the in-process runCommand/exitFunc doubles and httptest SSE streams, not subprocesses.
• Regression guard: existing tests that assert run() returns 0 on a completed stream (e.g. the happy-path tests in cmd/harnesscli/main_test.go) must keep passing unchanged; any test that was implicitly relying on "failed run exits 0" is a bug in the test and must be fixed in Slice 2, not preserved.
• e2e tests must exercise the real HTTP+SSE path (server + stream), including cancel-mid-run, per existing test/e2e patterns.
• Blocked-path tests (Slice 3) must cover all three blocked signals — run.waiting_for_user, tool.approval_required, plan.approval_required — both with non-interactive stdin (exit 3) and with terminal stdin simulated (no early exit), using an injectable terminal-detection double rather than a real TTY.
• Goal-status exit codes (3/6 reservation) are documentation-only in this epic: there is no goal-scoped CLI or HTTP surface to test against (internal/goals is reachable only via the in-run goals tool), so no runtime goal-mapping test is required — but the contract page must state this explicitly so a future goal epic knows to add them.
• No test may assert on stdout text to infer outcome where an exit code assertion is the point of the test.

Documentation Updates Required

• New website/docs/reference/exit-codes.md (the contract; Slice 1) — confirm whether it must be registered in website/sidebars.ts or is picked up automatically under website/docs/reference/_category_.json.
• website/docs/cli/harnesscli.md — per-command exit-code behavior for one-shot and continue (Slice 4).
• website/docs/cli/go-code-wrapper.md — a short headless-scripting note showing $? usage through the wrapper (Slice 4).
• website/docs/reference/cli-flags.md — exit-code notes for streaming flags where relevant (Slice 4).
• website/docs/reference/events-catalog.md — cross-link from terminal events to their exit codes (Slice 1 or 4).
• docs/logs/engineering-log.md entry per repo convention when the behavior change lands (Slice 2/3).
• Folder index updates per docs/runbooks/documentation-maintenance.md if any docs/ pages are touched.

Dependencies

• Parent: epic(roadmap): kimi-code parity program — workflow and tracking #803 (kimi-code parity epic tracker).
• No code dependencies on other epics. Blocked-state detection (Slice 3) assumes the server keeps a waiting run alive, which is current behavior (RunStatusWaitingForUser, internal/harness/types.go:337).
• The future interactive ask-user epic (wiring handleAskUserQuestion into the stream loop) will change Slice 3's TTY-stdin branch but must preserve exit 3 for non-interactive stdin; that constraint is recorded here so the two epics do not conflict.
• Full goal blocked/paused exit semantics depend on a future goal-scoped headless surface (goal HTTP route / CLI subcommand); this epic only reserves the codes and documents the intended mapping.