Skip to content

[Fix] End a Fast turn's inference once its closeout is delivered - #2030

Merged
mrubens merged 1 commit into
developfrom
fix/fast-abort-after-closeout
Sep 2, 2026
Merged

[Fix] End a Fast turn's inference once its closeout is delivered#2030
mrubens merged 1 commit into
developfrom
fix/fast-abort-after-closeout

Conversation

@mrubens

@mrubens mrubens commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Stacked on #2029 (diagnostics); retarget to develop once that merges.

Problem

After send_chat_reply posts a closeout (or a closeout reaction or ignore_event ends the turn), OpenCode still starts one more model request to consume the tool result. The prompt tells the model to do nothing after a closeout, but the request happens regardless: it re-sends the full context, produces text nobody sees, and holds the conversation's turn lock for as long as the provider takes. On the nightly this was 10–25 s per turn (inferenceDurationMs minus firstResponseDurationMs), during which any follow-up in the same conversation queues behind a turn that is already done.

Change

  • When a new assistant message starts while the current instruction is closed, the turn aborts the OpenCode prompt with a dedicated reason. The prompt runner's existing abort path issues session.abort server-side, so the model stops generating rather than just the HTTP request being dropped.
  • Before aborting, the handler waits for any in-flight native steer drain. A queued follow-up that reopens the turn (new instruction version) wins and the abort is skipped; the existing "steer after closeout" test covers that race.
  • The inference attempt treats this abort as success (the visible reply already went out) and returns an empty prompt text, which the closed turn ignores. No retry, no failure classification, no session invalidation, so the warm OpenCode session survives for the next turn.
  • Diagnostics gain abortedAfterCloseout on the log line and aborted_after_closeout on the settled-turn telemetry, so the saving is measurable against postReplyInferenceDurationMs from [Feat] Report OpenCode setup phases and model usage in Fast turn diagnostics #2029.

Follow-ups admitted while a turn runs are durable parent events delivered as fresh turns once the lock frees, so ending the turn earlier cannot strand one.

Verification

  • New test: closeout on the first assistant message, second assistant message starts, the prompt signal is aborted only then, the turn resolves to the closeout, postReply is called once, no failure is classified, and telemetry reports success with abortedAfterCloseout: true.
  • src/server/fast-agent: 356 tests pass. Typecheck, oxlint, knip, and pre-push clean.

@roomote-community

roomote-community Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

No new code issues found. See task

Reviewed a459478

@mrubens
mrubens force-pushed the fix/fast-abort-after-closeout branch from 097bb05 to b2f9d0c Compare September 2, 2026 03:32
Base automatically changed from feat/fast-turn-inference-diagnostics to develop September 2, 2026 03:36
After send_chat_reply posts a closeout (or a closeout reaction or
ignore_event ends the turn), OpenCode still starts another model request
to consume the tool result. That request produces text nobody sees, costs
a full prompt of tokens, and holds the conversation's turn lock for as
long as the provider takes; on the nightly that was 10-25 seconds per
turn, during which any follow-up in the same conversation queues.

Abort the prompt when a new assistant message starts while the current
instruction is closed, after letting an in-flight steer drain finish so a
queued follow-up that reopens the turn still wins. The prompt runner's
existing abort path cancels the OpenCode session server-side. The attempt
treats that abort as success because the visible reply already went out.
Diagnostics record abortedAfterCloseout.
@mrubens
mrubens force-pushed the fix/fast-abort-after-closeout branch from b2f9d0c to a459478 Compare September 2, 2026 03:37
@mrubens
mrubens merged commit 79b4eff into develop Sep 2, 2026
17 checks passed
@mrubens
mrubens deleted the fix/fast-abort-after-closeout branch September 2, 2026 04:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants