Skip to content

Suggestion generation blocks: superseded requests are never cancelled #77

Description

@alpha5611331

Problem

When a new suggestion supersedes one that is still streaming, the previous request is never actually cancelled - not in the service, not on the wire, and so not on the server either.

  1. stopRunningTasks() only flips a boolean, and it is checked before await reader.read() (suggestion-live.service.ts). A parked read never observes it.
  2. requestStream passes no signal to fetch (api/client.ts), unlike the non-streaming request() which does pass AbortSignal.timeout.
  3. The abandoned body gets releaseLock() but never cancel(). Undici documents this directly: "Failing to manually consume or cancel response bodies can lead to excessive connection usage, reduced performance, and potential stalls or deadlocks." Its bodyTimeout/headersTimeout default to 300s, so a stuck card sits in Pending for five minutes.

Because the client never disconnects, the server has nothing to react to and drains the full completion from the provider - so superseded requests keep consuming the RPM/TPM budget the new request needs.

Ctrl+Shift+F11 reported as "incredibly slow, or doesn't show at all"

That hotkey is the action/triggered suggestion path (hotkeys.ts), which has its own distinct causes:

  • Doesn't show at all. startGenerateSuggestion refuses without actionLockService.tryAcquire, and the only release is another function's finally, which cannot run while parked at await reader.read(). One stalled stream disables the hotkey for at least 300s - indefinitely if the socket never trips the timeout. The same lock gates F9 and F12, so one stall takes out all three. Nothing in stop() or clear() releases it. The lock is also acquired before the try, so a throw in the setup block leaks it permanently with no network call involved.
  • Incredibly slow. Captures are taken at full physical display resolution, up to 4 images, base64-inflated ~33% server-side, plus profile and context at up to 128,000 characters each, against a large model.

Two more ways suggestions stop silently

  • A stream that yields zero chunks leaves a live suggestion in Pending forever. State is only promoted to Loading inside if (value), and the terminal check is if (state === Loading) -> Success. No timeout rescues it because the stream ended rather than stalled. The action service has the mirror bug: it renders a blank Success card.
  • An orphaned mic partial gates every live suggestion. selfPartialTranscript is cleared only by a matching final; an ASR websocket that drops mid-utterance never sends one, so suggestions are suppressed until the candidate next finishes speaking - which can span several interviewer questions if they stay quiet.
  • startAssistant's catch never tears down transcription. If liveTranscriptionService.start() throws after electron.transcription.start() succeeded, isActive stays true while runningState goes back to Idle. Live suggestions keep working but F9/F11/F12 refuse forever.

Fix

  • Real AbortController cancellation wired through requestStream, with a resettable stall timer (not AbortSignal.timeout, which is a total deadline and would truncate long healthy generations).
  • reader.cancel() before releaseLock().
  • Release the action lock on every path, plus a max-hold backstop.
  • Resolve empty streams to a stated error instead of leaving a card unresolved.
  • Staleness guard on the orphaned mic partial, and instrumentation on the gate.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions