Skip to content

feat(sdk-ts): pause/WAITING + async-execution dispatch (parity with Python) — closes #726#731

Merged
AbirAbbas merged 5 commits into
mainfrom
feat/ts-sdk-pause-726
Jul 7, 2026
Merged

feat(sdk-ts): pause/WAITING + async-execution dispatch (parity with Python) — closes #726#731
AbirAbbas merged 5 commits into
mainfrom
feat/ts-sdk-pause-726

Conversation

@AbirAbbas

Copy link
Copy Markdown
Contributor

Summary

Closes #726. The control-plane pause/resume mechanism (pause(), WAITING status, pause-aware timeout accounting, multi-hop propagation) previously existed only in the Python SDK. This ports it to the TypeScript SDK.

The investigation surfaced that ctx.pause() alone is not enough: the TS SDK held the HTTP dispatch connection open for the entire reasoner, so a long pause would die at the control plane's synchronous dispatch ceiling. Python avoids this by fast-acking dispatch with 202 Accepted and reporting the result out-of-band. So this PR ports three layers:

1. Async-execution dispatch (on by default; asyncExecution: false to opt out)

A reasoner dispatched by the control plane (carrying X-Execution-ID) is acknowledged immediately with 202 { status: "processing", execution_id } and run detached; its terminal status is delivered via POST /executions/{id}/status. A pause-aware active-time watchdog (executionBudgetMs, default 2h) guarantees a terminal status even if a reasoner hangs. Requests without the header (direct callers / tests) keep the synchronous request/response contract.

2. Pause primitive — ctx.pause() / Agent.pause()

Transitions the execution to WAITING via request-approval, then blocks on a promise resolved by the always-on POST /webhooks/approval route when the control plane delivers the decision. Returns an ApprovalResult; times out to { decision: 'expired' } rather than throwing. Backed by a new PauseManager + PauseClock (src/agent/pause.ts), mirroring the Python _PauseManager/PauseClock.

3. Multi-hop propagation

The remote call() path now submits async and polls for the result; when an awaited child enters WAITING, the caller pushes its own execution to WAITING via notifyAwaiterStatus (awaiter-status endpoint) — so ancestors don't time out while a descendant legitimately waits.

The control-plane side was already fully language-agnostic (202 handling, /status, request-approval, awaiter-status, approval-response webhook + CP→agent callback); the entire change is in the TS SDK.

Behavioural note (reviewers please weigh in)

Async dispatch defaults on for full Python parity — this changes the dispatch behaviour for existing TS agents (202-ack + out-of-band result instead of a held connection). The control plane already handles both paths identically and Python has run this way in production. asyncExecution: false restores the legacy synchronous behaviour. Terminal results are wrapped so a reasoner returning a non-object still delivers cleanly.

Testing

  • Unit/integration: 35 new vitest cases; full suite 668 passing (was 633). Lint (tsc --noEmit) clean. Ran the exact CI gates (npm ci / npm run lint / npm test) on Node 20.
  • Live, end-to-end against a locally-built control plane (local SQLite/BoltDB storage) on a non-default port, with two agents (orchestrator + worker), all passing:
    1. Async dispatch → WAITING: submitting a pausing reasoner returns 202, the execution reaches waiting / waiting_for_approval — proving the dispatch connection was not held.
    2. Full CP loop: HMAC-signed approval-response → CP transitions to running and calls back the agent → reasoner resumes → terminal succeeded with the approval result.
    3. Multi-hop: the orchestrator cascaded to waiting / awaiting_child while the worker was paused (awaiter-status observed server-side), then resumed to succeeded with the child's result nested.

API additions

  • ctx.pause(opts) / agent.pause(opts)Promise<ApprovalResult>
  • ApprovalResult, PauseManager, PauseClock exported from the package root
  • AgentConfig.asyncExecution?: boolean (default true), AgentConfig.executionBudgetMs?: number
  • AgentFieldClient: executeAsync, getExecutionStatus, waitForExecutionResult, notifyAwaiterStatus, reportExecutionResult

🤖 Generated with Claude Code

AbirAbbas and others added 3 commits July 7, 2026 14:47
…th Python)

Ports the control-plane pause/resume mechanism to the TypeScript SDK
(closes the gap tracked in #726, where it existed only in the Python SDK).

Three layers:

1. Async-execution dispatch (on by default, `asyncExecution` config to opt out):
   a reasoner dispatched by the control plane (carrying `X-Execution-ID`) is
   acknowledged immediately with `202 Accepted` and run detached; its terminal
   status is delivered out-of-band via `POST /executions/{id}/status`. This
   frees the dispatch connection so a reasoner can wait far longer than the
   control plane's synchronous dispatch ceiling. A watchdog (pause-aware
   active-time budget) guarantees a terminal status even if a reasoner hangs.

2. Pause primitive: `ctx.pause()` / `Agent.pause()` transition the execution to
   WAITING via request-approval and block on a promise resolved by the always-on
   `POST /webhooks/approval` route when the control plane delivers the decision.
   Returns an `ApprovalResult`; times out to `{ decision: 'expired' }` rather
   than throwing. Backed by a PauseManager + PauseClock (new src/agent/pause.ts).

3. Multi-hop propagation: the remote `call()` path now submits async and polls
   for the result, and when an awaited child enters WAITING it pushes the
   caller's own execution to WAITING via `notifyAwaiterStatus` — so ancestors
   don't time out while a descendant legitimately waits.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- pause.test.ts: PauseClock accounting, PauseManager resolve/fallback/cancelAll,
  ApprovalResult getters, and the /webhooks/approval route.
- agent_async_execution.test.ts: 202 fast-ack + out-of-band succeeded/failed
  reporting, non-object result wrapping, sync fallback (no header /
  asyncExecution:false), and the reasoner_timeout watchdog.
- agent_pause.test.ts: end-to-end ctx.pause() resume via webhook, expired
  timeout, request-approval failure, and the awaiter-status multi-hop cascade,
  all driven through a mock control plane.
- agentfield_client_async.test.ts: executeAsync / getExecutionStatus /
  waitForExecutionResult (incl. WAITING-window clock pause) / notifyAwaiterStatus
  / reportExecutionResult.
- agent_runtime_paths.test.ts: pin the existing remote-delegation test to the
  synchronous path (asyncExecution:false); the async default is covered above.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds `planWithPause`, which uses the high-level `ctx.pause()` primitive
alongside the existing low-level ApprovalClient demo, so the example shows both
the parity API and the manual polling approach.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@AbirAbbas AbirAbbas requested a review from a team as a code owner July 7, 2026 18:48
@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Performance

SDK Memory Δ Latency Δ Tests Status
TS 463 B +32% 1.37 µs -31%

Regression detected:

  • TypeScript memory: 350 B → 463 B (+32%)

@AbirAbbas

Copy link
Copy Markdown
Contributor Author

Follow-up: verified the exact ceiling scenario from the issue (reproduce + fix)

Compressed the control plane's agent_call_timeout to 8s to simulate the dispatch ceiling, and ran two live agents against a local control plane:

Part A — reproduce the bug (sync dispatch, asyncExecution:false): a reasoner doing 20s of work (no pause mechanism) is dispatched synchronously and holds the connection. Result:

sync execute -> HTTP 504 after 8.0s
  {"error":"agent call failed: Post \".../reasoners/blockPastCeiling\": context deadline exceeded", "error_category":"agent_timeout"}

The call dies at the 8s ceiling — exactly the failure mode described in the issue.

Part B — prove the fix (async dispatch default + ctx.pause()): the reasoner is 202-acked and parks in WAITING. Polled every 2s, it stayed waiting / waiting_for_approval continuously across the full 20s (2.5× the ceiling), then an HMAC-signed approval resumed it:

t= 8.0s status=waiting   <- past the ceiling
t=12.0s status=waiting
t=20.0s status=waiting
approval-response -> new_status=running
TERMINAL status=succeeded result={"approved":true,"decision":"approved",...}

So the pause genuinely outlives the dispatch ceiling and resumes to a terminal succeeded — the sync path fails at the ceiling, the async+pause path survives it.

@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

📊 Coverage gate

Thresholds from .coverage-gate.toml: per-surface ≥ 84%, aggregate ≥ 85%, max per-surface regression ≤ 1.0 pp, max aggregate regression ≤ 0.50 pp.

Surface Current Baseline Δ
control-plane 86.90% 87.40% ↓ -0.50 pp 🟡
sdk-go 91.90% 92.00% ↓ -0.10 pp 🟢
sdk-python 93.76% 93.73% ↑ +0.03 pp 🟢
sdk-typescript 90.42% 90.42% → +0.00 pp 🟢
web-ui 84.75% 84.79% ↓ -0.04 pp 🟡
aggregate 85.54% 85.75% ↓ -0.21 pp 🟡

✅ Gate passed

No surface regressed past the allowed threshold and the aggregate stayed above the floor.

@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

📐 Patch coverage gate

Threshold: 80% on lines this PR touches vs origin/main (from .coverage-gate.toml:thresholds.min_patch).

Surface Touched lines Patch coverage Status
control-plane 0 ➖ no changes
sdk-go 0 ➖ no changes
sdk-python 0 ➖ no changes
sdk-typescript 268 94.00%
web-ui 0 ➖ no changes

✅ Patch gate passed

Every surface whose lines were touched by this PR has patch coverage at or above the threshold.

…Run-ID)

The functional test `test_ts_agent` invokes the reasoner via the legacy
synchronous endpoint `POST /api/v1/reasoners/{node}.{reasoner}`, which forwards
the agent's HTTP response verbatim and cannot handle a 202. With async dispatch
gated only on `X-Execution-ID`, the agent 202-acked there and the caller got the
`{status:"processing"}` marker instead of the result.

Gate async dispatch on BOTH `X-Execution-ID` and `X-Run-ID`. `X-Run-ID` is set
only by the control plane's async-aware `callAgent` path (workflow execute,
execute/async, agent-to-agent calls, triggers) — all of which wait for the
out-of-band `/status` result. The legacy sync invoke endpoint omits `X-Run-ID`
for long-running agents, so the agent now runs synchronously and returns the
result inline there, while pause/async continues to work on the execute paths.

Verified live: the legacy endpoint returns the inline echo result again, and a
pause submitted via execute/async still reaches WAITING and resumes to
succeeded. Adds a regression test for the X-Execution-ID-without-X-Run-ID case.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@AbirAbbas

Copy link
Copy Markdown
Contributor Author

CI fix: async dispatch now gated to async-aware paths (functional tests green)

The functional job caught a real regression from async-on-by-default. test_ts_agent invokes the reasoner via the legacy synchronous endpoint POST /api/v1/reasoners/{node}.{reasoner}, which forwards the agent's HTTP response verbatim and cannot handle a 202 — so the agent's {status:"processing"} marker was returned instead of the result.

Root cause: I gated async dispatch on X-Execution-ID alone, which that endpoint also sets. (Notably the Python SDK has the same latent behaviour on this endpoint — it's masked in CI by Docker layer-caching of the pip install sdk/python step when the Python SDK is unchanged.)

Fix: gate async dispatch on both X-Execution-ID and X-Run-ID. X-Run-ID is set only by the control plane's async-aware callAgent path (workflow execute, /execute/async, agent-to-agent calls, triggers) — all of which wait for the out-of-band /status result. The legacy sync invoke endpoint omits it for long-running agents, so the agent runs synchronously and returns the result inline there.

Verified live against a local control plane:

  • Legacy POST /api/v1/reasoners/tsfix.echoresult: {echoed:"hello-ts", runId, workflowId} (inline, 200).
  • Pause via /execute/async202WAITING → approve → terminal succeeded.

Added a regression test for the X-Execution-ID-without-X-Run-ID case. Full suite 669 passing; Functional Tests (local + postgres) and all TS gates green.

@AbirAbbas AbirAbbas merged commit 1e11e82 into main Jul 7, 2026
24 checks passed
@AbirAbbas AbirAbbas deleted the feat/ts-sdk-pause-726 branch July 7, 2026 19:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant