Skip to content

fix(cli): isolate model cache refresh from caller cancellation#12203

Merged
marius-kilocode merged 1 commit into
mainfrom
fix-model-cache-race
Jul 14, 2026
Merged

fix(cli): isolate model cache refresh from caller cancellation#12203
marius-kilocode merged 1 commit into
mainfrom
fix-model-cache-race

Conversation

@marius-kilocode

Copy link
Copy Markdown
Collaborator

Problem

A new Agent Manager worktree session can fail before its first model response with All fibers interrupted without error when automatic branch-name generation runs concurrently and the Kilo model cache is expired.

Branch-name generation has a 10-second timeout. When the cache is stale, both the new session and branch-name generation become concurrent waiters on the same shared model refresh. When the branch-name timeout fires, it interrupted the shared refresh, and every waiter on that refresh received an interrupt-only Effect cause. SessionPrompt.getModel squashed that cause into a generic error, which the asynchronous promptAsync handler published as UnknownError.

Root cause

The model cache used Effect.cachedInvalidateWithTTL, which coupled the shared refresh lifetime to the first caller. A timeout or interruption from one waiter could interrupt the computation observed by every waiter.

Fix

Move the in-flight refresh to the ModelCache service scope instead of the first caller:

  • A refresh fiber is forked into the service scope, not the caller.
  • Each caller awaits the shared deferred interruptibly.
  • Caller interruption detaches only that waiter; the service-owned refresh continues for remaining callers.
  • The service-owned fiber commits successful results even when no waiter survives, so a timed-out branch-name request no longer leaves the session without a model.
  • Overlapping refreshes share one request instead of duplicating.
  • Cached stale-version results are promoted to the public provider view when a newer option load has superseded and failed.
  • clear and invalidate still prevent obsolete flights from restoring stale data, and existing version checks still prevent stale refreshes from overwriting newer values.
  • Service disposal still interrupts owned background refresh fibers.

Treat pure interruption as cancellation rather than an error:

  • SessionPrompt.getModel propagates interrupt-only causes as interruption instead of squashing them into a generic defect.
  • The promptAsync HTTP handler suppresses interrupt-only causes instead of publishing UnknownError with All fibers interrupted without error.
  • Mixed interruption plus failure or defect remains reportable.

Classification lives in a Kilo-owned helper (packages/opencode/src/kilocode/effect/cause.ts) using Cause.hasInterruptsOnly, so only pure interruption is treated as cancellation.

Tests

  • keeps a shared refresh alive when one waiter times out: one caller times out; a second caller for the same provider still receives the model list from the single shared request.
  • commits a refresh after its only waiter times out: a service-owned refresh commits to the public cache even when its only waiter has already detached.
  • deduplicates overlapping refresh calls: two concurrent explicit refreshes share one in-flight request.
  • promotes a cached result after a newer option load fails: a superseded option-specific result restores the public provider view when the newer load fails.
  • retries after a failed refresh: a failed load is not cached so the next call retries.
  • Existing option isolation, stale-refresh ordering, and clear-while-pending tests are retained and pass.
  • effect-cause.test.ts covers pure interruption, a defect, and a mixed interrupt-plus-defect cause.

A new Agent Manager worktree session can fail before its first model
response with "All fibers interrupted without error" when automatic
branch-name generation runs concurrently and the Kilo model cache is
expired. Branch-name generation has a 10-second timeout; when it fires,
it interrupted the shared cached model refresh, and every waiter on that
refresh received an interrupt-only cause. SessionPrompt.getModel squashed
that cause into a generic error, which promptAsync published as
UnknownError.

The model cache previously used Effect.cachedInvalidateWithTTL, which
coupled the shared refresh lifetime to the first caller. A timeout or
interruption from one waiter could interrupt the computation observed by
every waiter.

Move the in-flight refresh to the ModelCache service scope:
- A refresh fiber is forked into the service scope, not the caller.
- Each caller awaits the shared deferred interruptibly.
- Caller interruption detaches only that waiter; the service-owned
  refresh continues for remaining callers.
- The service-owned fiber commits successful results even when no waiter
  survives, so a timed-out branch-name request no longer leaves the
  session without a model.
- Overlapping refreshes share one request instead of duplicating.
- Cached stale-version results are promoted to the public provider view
  when a newer option load has superseded and failed.
- Clear and invalidate still prevent obsolete flights from restoring
  stale data, and existing version checks still prevent stale refreshes
  from overwriting newer values.
- Service disposal still interrupts owned background refresh fibers.

Treat pure interruption as cancellation rather than an error:
- SessionPrompt.getModel propagates interrupt-only causes as interruption
  instead of squashing them into a generic defect.
- The promptAsync HTTP handler suppresses interrupt-only causes instead
  of publishing UnknownError with "All fibers interrupted without error".
- Mixed interruption plus failure or defect remains reportable.

Classification lives in a Kilo-owned helper
(packages/opencode/src/kilocode/effect/cause.ts) using
Cause.hasInterruptsOnly, so only pure interruption is treated as
cancellation.
@marius-kilocode
marius-kilocode disabled auto-merge July 14, 2026 10:21
@marius-kilocode
marius-kilocode merged commit bb1faec into main Jul 14, 2026
29 of 30 checks passed
@marius-kilocode
marius-kilocode deleted the fix-model-cache-race branch July 14, 2026 13:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants