adapter-codex: parse "tokens used" footer into Cost event#404
Merged
Conversation
The codex CLI prints a per-turn footer:
tokens used
2,280
on two consecutive stdout lines. Until now the adapter forwarded both lines
as assistant `Message` events, polluting the transcript and leaving the
session with no `Cost` event for downstream analyzers (token counts came
back blank in our swebench gap reports).
spawn_stdout now tracks a two-line state machine: when a line equals
"tokens used", the next line is parsed as a comma/underscore-tolerant
integer and emitted as `SessionEvent::Cost { tokens_in: N, ... }`. Both
the header and the count are swallowed (no longer leak to Message).
codex reports a single "total tokens" per turn without splitting in/out,
so we store the total under `tokens_in` as a conservative proxy — the
prompt/context dominates and downstream analyzers can still distinguish
"have token info" from "don't" (tokens_out stays 0).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tokens used\n<N>on two stdout lines; the adapter was forwarding both as assistantMessageeventsCostevent was ever emittedspawn_stdoutruns a two-line state machine — when a line equalstokens used, the next line is parsed as a comma/underscore-tolerant integer and emitted asSessionEvent::Cost { tokens_in: N, ... }. Both lines are swallowed so they no longer leak toMessage.codex reports a single total per turn (no in/out split), so the total goes into
tokens_inas a conservative proxy — downstream analyzers can still distinguish "have tokens" from "don't" (tokens_outstays 0).Test plan
cargo test --manifest-path crates/adapter-codex/Cargo.toml— 7 passed including newparse_token_count_handles_codex_formatscargo build --release --manifest-path crates/adapter-codex/Cargo.toml— cleanswebench-zarvis/runner/analyze.pyreports non-zerotokens_in