Skip to content

feat(runtime): stop a turn on the clock as well as the step count - #64

Merged
mangit955 merged 1 commit into
mainfrom
feat/wall-clock-budget
Aug 30, 2026
Merged

feat(runtime): stop a turn on the clock as well as the step count#64
mangit955 merged 1 commit into
mainfrom
feat/wall-clock-budget

Conversation

@mangit955

Copy link
Copy Markdown
Owner

Implements #60 (already closed). ADR committed alongside the code at docs/adr/0001-wall-clock-budget-for-the-agent-loop.md.

Why

The loop has only ever measured its budget in iterations while the harness enforces a wall clock. make-mips-interpreter was killed by its own 200th iteration at 406s of an 1800s budget, mid-work, with exception_info: null proving Harbor's timeout never fired. Across the five-task config every trial spent 2–23% of the time it was given.

What

  • runtime/deadline.ts — module singleton in the shape of runtime/sandbox/registry.ts. WALL_RESERVE_SEC = 60 held back so the last step, the final answer and the session write land before the harness kills the process. The clock is injectable, which is how elapsed time is tested without stubbing Date globally.
  • runtime/loop.tsBudgetExhaustedError base with IterationBudgetExhaustedError and a new WallBudgetExhaustedError as siblings; WOOPCODE_MAX_WALL_SEC read once per turn; the deadline checked at the top of an iteration, inside the try, so it takes the same onError-then-rethrow path.
  • The wind-down is unified. Remaining time ÷ the turn's own measured wall-per-iteration gives a step count, so one message and one flag serve both budgets.
  • commands/agent.tsx — one instanceof BudgetExhaustedError in place of two, so both budgets exit 2. A distinct code would be booked as an exception by any harness not yet updated to know it.

onBudgetExhausted is deliberately not consulted for the deadline: the step ceiling can afford to ask because iterations do not tick while a human thinks, and a clock does.

Behaviour change

The wind-down flag replaces an equality test that silently never fired when the ceiling was below five steps — the turns with least room were the ones told nothing. A two-step budget is now announced at its first step. iterationBudget.test.ts was updated to assert the new behaviour.

Verification

  • bun run verify --all — 4 gates passed (docs lint, docs surface, type check, tests).
  • bun run replay:baselinebyte-identical before and after, across all ten fixtures and the corpus totals. Expected: this decides when a turn stops, not what the prompt carries.
  • Six mutations applied, confirmed present in the file, run red, then restored: the deadline throw (3 fail), clearDeadline in finally (1), the wind-down reset (1), the wall term in stepsRemaining (3), the reserve subtraction (11), the clamp floor (1). The first disarm test survived its mutation and was rewritten to assert remainingMs() directly.
  • Reverse-order sweep and each affected file in isolation, both green.

Not checked: a live Harbor run, a live provider, interactive TUI input.

Deferred, and load-bearing

This makes the failure fixable, not fixed. Both of these are needed before it stops happening:

Two things a reviewer should weigh

Both were raised in review and left as the ADR decided them, rather than redesigned here:

  1. WOOPCODE_MAX_WALL_SEC in a .env would break the TUI. Bun auto-loads .env, the deadline anchors to process start, and the interactive path skips onBudgetExhausted — so after the budget elapses every turn throws before its first request, with no checkpoint. The ADR's reasoning rests on "the interactive path will not have the variable set," which the code does not enforce.
  2. meanStepMs is a whole-turn mean and the warning flag is permanent. One slow step can tell the model to wrap up while plenty of budget remains, and it is never taken back. It nudges rather than ends the turn.

The loop has only ever measured its budget in iterations while the harness
that runs it enforces a wall clock, so `make-mips-interpreter` was killed by
its own 200th iteration at 406s of an 1800s budget, mid-work, with
`exception_info: null` proving Harbor's timeout never fired. Across the
five-task config every trial spent 2-23% of the time it was given.

`WOOPCODE_MAX_WALL_SEC` gives the loop a second budget and it stops on
whichever binds first. The operator passes the whole budget; `setDeadline`
subtracts a 60s reserve so the last step, the final answer and the session
write still land. The deadline lives in module state for the reason
`runtime/sandbox/registry.ts` gives for the same shape.

`onBudgetExhausted` is deliberately not consulted for the deadline: the step
ceiling can afford to ask because iterations do not tick while a human
thinks, and a clock does. `WallBudgetExhaustedError` is a sibling of
`IterationBudgetExhaustedError` under a shared `BudgetExhaustedError`, so both
exit 2 - one code, because a distinct one would be booked as an exception by
any harness not yet updated to know it.

The wind-down now converts remaining time into steps at the turn's own
measured rate, so one message and one flag serve both budgets. That flag
replaces an equality test that silently never fired when the ceiling was
below five, which is a behaviour change: a two-step budget is now announced
at its first step.

Prompt assembly is untouched - `bun run replay:baseline` is byte-identical
before and after, across all ten fixtures and the corpus totals.

Reasoning and the per-task measurements:
docs/adr/0001-wall-clock-budget-for-the-agent-loop.md.

Deferred by design: clamping tool timeouts to the remaining budget (#61) and
wiring the budget through `agent.py`, `job.yaml` and CLAUDE.md (#62). Until
#61 lands the deadline is advisory for a command already running.

Closes #60
@vercel

vercel Bot commented Aug 30, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
woop-code Ready Ready Preview Aug 30, 2026 3:53am

@mangit955
mangit955 merged commit a0f2580 into main Aug 30, 2026
6 checks passed
@mangit955
mangit955 deleted the feat/wall-clock-budget branch August 30, 2026 03:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant