fix(chat): surface the fallback when a turn ends on a tool-call step - #55
Merged
Conversation
The empty-turn guard keyed on `producedText`, which latches on the first word of a turn and never resets. A lead agent that announces a delegation and calls its sub-agents in the same step therefore looks, at the end of the turn, exactly like one that answered: the guard stays quiet and the user is left with settled chips, no answer and no error. Observed on a dataset-metadata turn: two sub-agents finished cleanly server-side (both `finishReason: stop`), the lead never issued another request, and the chat simply became available again — one turn in three during that campaign. Track instead whether the step that just finished issued a tool call. A turn whose last step called a tool is one the model meant to continue: it had a result to read and said nothing. That also covers the case the original comment already named, the step limit being reached on a tool call.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A turn can end with no answer, no error and no fallback. Observed once in three during a traced audit of the assistant on dataset metadata: the lead announced a delegation, launched two sub-agents, both finished cleanly server-side (
finishReason: stopon each), and the lead never issued another request. The chips settled, nothing was appended, the input became available again. From the user's side the assistant simply stopped mid-thought — and in that particular turn, 2 columns out of 22 had been configured and nothing else.Why the existing guard missed it
use-agent-chat.tsalready protects against silent drops:producedTextlatches on the first text delta of the turn and never resets. The model almost always announces what it is about to do in the very step that calls the sub-agents — here, « Je délègue l'analyse approfondie. » — so by the end of the turnproducedTextis true and the guard stays quiet. A turn that spoke once and then died is indistinguishable from a turn that answered.The fix
Track whether the step that just finished issued a tool call, and fall back on that too:
A turn whose last step called a tool is a turn the model meant to continue: it had a result to read and said nothing. That is true whatever it said earlier, which is what the text-based signal could not express.
It also covers, explicitly, a case the original comment already named but could not catch when the turn had spoken:
stopWhen: stepCountIs(10)reached on a tool call.lastStepHadToolis set atfinish-stepfrom the existingstepHadTool, which was already tracked for the activity label — no new stream bookkeeping.Testing
5 new unit tests in
tests/features/empty-turn/, pinning the distinction the fix rests on — in particular that text and the sub-agent calls landing in one step still counts as a turn that never came back, which is the exact shape of the observed failure and the case my first attempt (a text-based signal) got wrong. 339 unit tests pass; lint andcheck-typesclean.What is not verified
The e2e suite has not been run — it needs the dev stack up. Two things a reviewer may want to exercise before merging:
chat-silent-dropandchat-subagent, for regressions on the paths that already surface a fallback;There is a mock seam for this shape already (
loop forever, used bychat-subagentfor the sub-agent close-out path), so an e2e for the main-agent case would be cheap to add if you want it in this PR.