[Fix] End a Fast turn's inference once its closeout is delivered - #2030
Merged
Conversation
Contributor
mrubens
force-pushed
the
fix/fast-abort-after-closeout
branch
from
September 2, 2026 03:32
097bb05 to
b2f9d0c
Compare
Base automatically changed from
feat/fast-turn-inference-diagnostics
to
develop
September 2, 2026 03:36
After send_chat_reply posts a closeout (or a closeout reaction or ignore_event ends the turn), OpenCode still starts another model request to consume the tool result. That request produces text nobody sees, costs a full prompt of tokens, and holds the conversation's turn lock for as long as the provider takes; on the nightly that was 10-25 seconds per turn, during which any follow-up in the same conversation queues. Abort the prompt when a new assistant message starts while the current instruction is closed, after letting an in-flight steer drain finish so a queued follow-up that reopens the turn still wins. The prompt runner's existing abort path cancels the OpenCode session server-side. The attempt treats that abort as success because the visible reply already went out. Diagnostics record abortedAfterCloseout.
mrubens
force-pushed
the
fix/fast-abort-after-closeout
branch
from
September 2, 2026 03:37
b2f9d0c to
a459478
Compare
daniel-lxs
approved these changes
Sep 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #2029 (diagnostics); retarget to
developonce that merges.Problem
After
send_chat_replyposts a closeout (or a closeout reaction orignore_eventends the turn), OpenCode still starts one more model request to consume the tool result. The prompt tells the model to do nothing after a closeout, but the request happens regardless: it re-sends the full context, produces text nobody sees, and holds the conversation's turn lock for as long as the provider takes. On the nightly this was 10–25 s per turn (inferenceDurationMsminusfirstResponseDurationMs), during which any follow-up in the same conversation queues behind a turn that is already done.Change
session.abortserver-side, so the model stops generating rather than just the HTTP request being dropped.abortedAfterCloseouton the log line andaborted_after_closeouton the settled-turn telemetry, so the saving is measurable againstpostReplyInferenceDurationMsfrom [Feat] Report OpenCode setup phases and model usage in Fast turn diagnostics #2029.Follow-ups admitted while a turn runs are durable parent events delivered as fresh turns once the lock frees, so ending the turn earlier cannot strand one.
Verification
postReplyis called once, no failure is classified, and telemetry reports success withabortedAfterCloseout: true.src/server/fast-agent: 356 tests pass. Typecheck, oxlint, knip, and pre-push clean.