TTFB watchdog token estimator reports ~1.9M tokens while API prompt_tokens stays ~90-175K (extreme-aspect images / large data-URI tool output)

[dombejar]

Summary

The context-size estimate logged by the openai-codex no-byte TTFB watchdog (Disabling openai-codex no-byte TTFB watchdog for large request (context=~X tokens ...)) can balloon far past reality. In one session (v0.18.2, gpt-5.6-sol via openai-codex) it climbed to ~1.9M tokens, then dropped abruptly to ~689K and climbed back toward ~750K — while the authoritative per-call accounting (API call #N ... in= / prompt_tokens) for the same session stayed sane the whole time (90K-175K), and the real compression trigger path behaved normally.

Suspected trigger

The divergence began after the session processed:

• two vision_analyze calls on browser screenshots auto-resized to extreme aspect ratios (195x5834 and 195x6042 px), and
• tool output containing large embedded SVG / data: URI payloads.

So the char-based (or image-cost) estimator on the watchdog path appears to massively overcount these payloads relative to what is actually sent/tokenized.

Impact

Cosmetic-plus: the real compression gate keys off real token accounting so no misbehavior was observed there, but the log line is alarming during incident forensics (a phantom "1.9M-token session"), and anything else that ever consumes this estimate would inherit the error.

Environment

hermes-agent 0.18.2 (at af250d8), provider openai-codex, model gpt-5.6-sol, Linux.

https://claude.ai/code/session_015StmTvr85VriETdW9Gy9D5

[GottZ]

This was generated by AI during triage.

Summary

One open PR addresses issue #63871. #63913 replaces raw character counting with the shared image-aware estimator for message-like Chat and Responses inputs, reducing base64-driven inflation, but it does not cover images nested in Codex function_call_output.output.

Related pull requests

• fix(agent): reduce TTFB watchdog token over-estimation for image-heavy turns (#63871) #63913 best fix — (+23/-15) — partial fix: Reusing estimate_messages_tokens_rough bounds ordinary image-content estimates instead of charging for raw base64, directly addressing one reported cause. Consistent with the keep_open review on fix(agent): reduce TTFB watchdog token over-estimation for image-heavy turns (#63871) #63913, the diff does not extend the estimator to Codex multimodal tool results under function_call_output.output and adds no regression tests for either payload form.

Suggested consolidation

Merge #63913 after extending the estimator to cover image arrays in Codex function_call_output.output and adding bounded-estimate regression tests for ordinary message content and Responses tool output; there are no duplicate PRs to close.

Complex graph

flowchart LR
    classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
    classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
    classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
    classDef best stroke-width:3px,stroke:#b45309
    classDef target stroke-width:3px,stroke:#4338ca
    I63871(["issue #63871 (open)"])
    P63913["PR #63913 (open)"]
    P63913 -->|best fix| I63871
    class I63871 open
    class I63871 target
    class P63913 open
    class P63913 best
    click I63871 "https://github.com/NousResearch/hermes-agent/issues/63871"
    click P63913 "https://github.com/NousResearch/hermes-agent/pull/63913"

Loading

Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed or no verify verdict yet (state tag in the node label).

Cross-PR triage: Reviewed 1 pull request and 1 issue in this complex. Each diff was read against this issue; Assessment working set: 3 kB of PR diffs, 2 kB of issue/PR text, 1 kB of discussion (1 comments), 2 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.