Stream by default, and recognize Copilot's unauthenticated wording - #11
Merged
Conversation
added 2 commits
July 29, 2026 07:48
The auth classifier's Copilot branch was inference, not observation. I had said
so, because a bogus COPILOT_GITHUB_TOKEN still fell through to the stored
credential and I could not produce a logged-out run to look at.
Pointing Copilot at an empty HOME produces one, and it turns out none of the
phrases matched:
Error: No authentication information found.
Claude says "Not logged in", Codex says "not logged in", Copilot shares no
vocabulary with either. So a missing Copilot login was reported as a generic
failure, with no login hint, which is exactly the case Error::NotAuthenticated
exists for. Its verbatim wording is now in the phrase list and pinned by a test.
Worth noting what this says about the other entries: the phrases that were
observed are right and the one that was inferred was wrong, in a way no amount
of reasoning about it would have caught. The remaining unobserved phrases are
still guesses, and they stay only because a false negative there costs a worse
error message rather than a wrong result.
Watching a run was not possible with the settings a caller gets for free, which made the streaming support largely theoretical. Two causes. Format defaulted to Json, under which nothing is observable until the turn ends: a twenty-minute run reported nothing for twenty minutes, which is indistinguishable from a hang. And Request::session *pinned* that format, so the multi-turn path a chat UI always uses could not stream at all, which is the worst place for it to be off. Stream is now the default. It carries everything Json does, the session id and any schema-conforming value included, verified against the live CLI, so the change costs only parsing. session() now validates the format rather than overriding it, which is all it needed to do: the default already carries an id, and its only real job is refusing a format that cannot. Claude also needed --include-partial-messages, without which it emits only completed messages and text arrives a paragraph at a time. With it, Claude streams tokens like Copilot already did, so the two behave alike rather than the abstraction making Claude the coarser of the two. Claude sends both the deltas and the completed message they build up to. Emitting both would show every answer twice, so the completed copy is dropped once deltas have been seen. Detected rather than configured: deltas always precede the finished message, so seeing one proves the copy is redundant. Suppression applies to text and thinking only, never to tool calls, which the deltas do not duplicate. Proven live rather than by flag inspection: a real run streams 4 chunks for a 254-byte answer with no format specified, and the concatenated chunks do not exceed the final answer, which is what would happen if the finished message were emitted on top of them.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The two commits that missed #10, which was merged at its first commit. Both are complete and were verified live before this PR existed.
1. Streaming was effectively off by default
Watching a run was not possible with the settings a caller gets for free.
Formatdefaulted toJson, under which nothing is observable until the turn ends: a twenty-minute run reported nothing for twenty minutes, which is indistinguishable from a hang. Worse,Request::sessionpinned that format, so the multi-turn path a chat UI always uses could not stream at all.That was my doing. I chose
Jsonbecause it was the cheaper format that carried a session id, which optimised for the wrong thing.Streamis now the default, verified against the live CLI to carry everythingJsondoes (session_idandstructured_outputincluded), so the change costs only parsing.session()now validates the format instead of overriding it, which is all it ever needed to do.Claude also needed
--include-partial-messages; without it it emits only completed messages and text arrives a paragraph at a time. With it, Claude streams tokens the way Copilot already did — the abstraction had been making Claude the coarser of the two, which is backwards.Claude sends both the deltas and the finished message, so the finished copy is dropped once deltas are seen. Detected rather than configured, since deltas always precede it. Suppression covers text and thinking only, never tool calls.
Proven live:
streamed 4 chunks for a 254 byte answer, with no.format()call. The test also asserts the chunks do not exceed the final answer, which is what would happen if the finished message were emitted on top of them.2. Copilot's unauthenticated wording
Its branch of the auth classifier was inference, which I had flagged as unverified. Pointing Copilot at an empty
HOMEproduces a real logged-out run, and none of the phrases matched:Claude says "Not logged in", Codex says "not logged in", Copilot shares vocabulary with neither. So a missing Copilot login was reported as a generic failure with no login hint. Now pinned by a test using the verbatim string.
Worth noting: the phrases that were observed were all correct, and the one that was inferred was wrong. The remaining unobserved phrases are still guesses.
Verification
Rebased onto current master and re-verified: 117 unit tests, clippy clean under
-D warnings,cargo packageverifies, full live suite 14/14 against all three real agents.Process note
An earlier version of this said follow-up commits kept being stranded by merges landing first. That was wrong, and unfair to whoever merged.
The timestamps say the opposite:
Those commits did not lose a race. They did not exist yet. Both PRs were already merged and closed, and I kept pushing new work to their branches, where it could never land regardless of anyone else'"'"'s timing.
The actual fault is mine and it is mundane: I never checked PR state before pushing more commits to its branch. One
gh pr view --json statecatches it every time. Hence this PR being opened complete, with nothing added afterwards.