Repository navigation
Adr 0002 Runtime Execution Boundary
Status: #142 merged via #147; #143 merged via #156; #144 merged via #161. The #145 retry/fallback policy is proposed, not yet claimed as merged. Parent: #94 (AG9 Model Runtime).
The first AG9 slice forwards cancellation to cooperative providers and bounds the entire request with one absolute deadline. It does not implement real streaming, configurable retries/chains, new providers or skill-specific fallback redesign.
Implementation sequence: #142 cancellation/deadline → #143 budgets → #144 real streaming → #145 retry/fallback policy → #146 deterministic skill fallback and final conformance audit. Tests ship with every slice. Landing one slice does not close #94.
The merged #143 budget boundary applies one frozen, bounded request across text/structured/fragment paths and retries/fallback. It rejects oversized whole-input text without clipping the question and separates character counts, per-completion token caps and optional reported usage. Budget values/reports remain JSON data; execution controls remain local.
CompletionRequest and RuntimeRequestData remain JSON-safe data. Neither receives an
AbortSignal, callback, controller or executable provider object. Signals remain in
ProviderExecutionOptions / RuntimeExecutionOptions in @focusloop/llm-provider.
ExecutableAIProvider extends the existing vocabulary contract with an optional process-local
second argument. Existing one-argument providers remain structurally compatible. The runtime races
non-cooperative providers too, but cannot forcibly release resources a custom adapter refuses to
release; cooperative transports must consume the supplied signal.
| Cause | Text API | Structured API | Further provider work |
|---|---|---|---|
| Caller cancellation | rejects with AbortError
|
aborted, no value/commit |
none |
| Total runtime deadline reached | rejects with timeout ProviderError
|
expired, no value/commit |
none |
| Provider-local timeout/error while runtime time remains | bounded fallback | bounded, schema-validated fallback | existing fallback only |
Provider-local AbortError with the caller's signal untouched (an adapter bounding its own transport) |
bounded fallback | bounded, schema-validated fallback | existing fallback only |
Equality counts as expired: Date.now() >= deadlineMs. A deadline is absolute epoch milliseconds,
not a duration restarted on retry/fallback. Non-finite deadlines are rejected before work starts.
Caller cancellation takes precedence when both boundaries are observed. A primary-local timeout is
not the same as runtime expiration; DeepSeek normalizes its own timer to the existing provider
timeout failure classification.
Only the caller's own cancellation is terminal. Cancellation is decided by ExecutionAbortError
(raised when options.signal is aborted) or RuntimeDeadlineError, never by matching
error.name === 'AbortError' on whatever a provider threw: an AbortError a provider authored for
itself is a provider failure and still degrades to the deterministic fallback, on both the text and
the structured path. The fallback call is not wrapped in a second catch, so a fallback that itself
throws rejects the text call instead of being reported as degraded; the shipped deterministic
fallback cannot throw, so that is a documented boundary rather than an exercised path.
The linked execution signal reaches fetch and response-body consumption. Each execution releases its timer and caller abort listener on success, error, cancellation or expiration. Timer delays above Node's maximum are re-armed instead of being clamped to a premature 1ms expiration. A late result or rejection from an adapter that ignores the signal is observed but never accepted or used to start fallback.
Check cancellation/deadline before every attempt, after awaited completion, and after schema validation immediately before accepting/committing a value. Validation and the commit invocation have no intervening asynchronous boundary. The hook is a synchronous caller-owned action; this contract does not cancel asynchronous work initiated by that hook.
A throwing commit is a caller failure, not a model/provider failure: propagate it rather than retry, invoke fallback, or commit twice. Invalid primary output still gets at most one schema retry, followed by one fallback. A fallback's output must validate before any commit; generic mock prose is not magically valid structured JSON.
The original #142 compatibility stream entry points used collected completions. The merged #144 incremental streaming boundary consumes optional process-local provider iterables, with an honest collected compatibility adapter when absent. First display text may precede response-body completion; after-first-text failure cannot splice in fallback. Structured streams must reach a protocol-complete, schema-validated final result before committing. The same absolute deadline governs reads, idle consumer pauses and retries; cancellation ends the string view quietly, expiration rejects, and abandonment releases cooperative readers without waiting for a pending next. Renderer streaming IPC is not part of these slices.
The proposed #145 retry/fallback policy extends, without replacing,
the primary → local-fallback contract. Retryable reasons (offline, timeout, rate-limited) may
repeat the same provider inside an injected, capped attempt limit; unauthorized, not-configured,
bad-response, cancellation and the deadline never retry. One ledger bounds primary transport
retries, schema retries and fallback calls together (maxModelCalls), and the primary reserves the
last slot so the deterministic local fallback is always reachable. Pauses are injected, deterministic
and refuse to cross the absolute deadline; no attempt or pause starts after cancellation/expiration.
Provenance is reported per attempt. Streaming keeps the split above: pre-text failures may retry or
fall back, post-text failures never splice a second provider's answer.
The first RED run against upstream main recorded 11 failed / 37 passed in
pnpm --filter @focusloop/llm-provider test: no forwarded signal, cancellation-triggered fallback,
late deadline work/commit, and DeepSeek caller cancellation misclassified as timeout. Those counts
are from the first batch of tests; the final suite is larger. Repeating the run against 1cfb124
with the final suite fails 20 of the 29 assertions in the two new files.
Test power, stated precisely: the seven execution.spec.ts resource-ownership cases unit-test the
new execution.ts directly, so they cannot fail against main - they are new-code unit tests, not
regression evidence. The regression evidence is the 20 failures above.
The GREEN provider suite includes both existing conformance tests and new fake-provider/fetch/time fixtures for primary/fallback cancellation, deadlines before work and during retry/fallback, response-body cancellation, non-cooperative late results, resource cleanup, long timers, validation boundaries and failed commit hooks. No real API credentials, external model calls, prompt logs, production telemetry or new dependencies are introduced.
Verify with pnpm --filter @focusloop/llm-provider test, repository test/typecheck/lint/build, and
pnpm e2e. The offline golden path remains the product integration gate. This slice exposes the
process-local controls but does not add a renderer cancel button or streaming IPC channel.