Show per-request tokens in the visualizer

[enyst]

This PR proposes to display in the default visualizer the tokens for the last request, not the accumulated values. I'd find it much more useful to keep an eye on the context window.

(The accumulated input tokens get into millions fast, and then they don't mean much really?)

IMHO this makes the default visualizer more useful, for tracking context window, reasoning tokens, what happened each request.

Tested and this is what it looks like:

Summary

• surface token counts for the most recent LLM request in the visualizer subtitle
• remove the accumulated token view and highlight only the current call
• keep cumulative cost but label it as "(total)" for clarity

Testing

• uv run pytest tests/sdk/conversation/test_visualizer.py
• uv run pre-commit run --files openhands/sdk/conversation/visualizer.py tests/sdk/conversation/test_visualizer.py

Fixes #803

---

Agent Server images for this PR

• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant	Base Image	Docs / Tags
golang	golang:1.21-bookworm	Link
java	eclipse-temurin:17-jdk	Link
python	nikolaik/python-nodejs:python3.12-nodejs22	Link

Pull (multi-arch manifest)

docker pull ghcr.io/openhands/agent-server:e3c5206-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-e3c5206-python \
  ghcr.io/openhands/agent-server:e3c5206-python

All tags pushed for this build

ghcr.io/openhands/agent-server:e3c5206-golang
ghcr.io/openhands/agent-server:v1.0.0a3_golang_tag_1.21-bookworm_binary
ghcr.io/openhands/agent-server:e3c5206-java
ghcr.io/openhands/agent-server:v1.0.0a3_eclipse-temurin_tag_17-jdk_binary
ghcr.io/openhands/agent-server:e3c5206-python
ghcr.io/openhands/agent-server:v1.0.0a3_nikolaik_s_python-nodejs_tag_python3.12-nodejs22_binary

The e3c5206 tag is a multi-arch manifest (amd64/arm64); your client pulls the right arch automatically.

[github-actions]

[Automatic Post]: I have assigned @neubig as a reviewer based on git blame information. Thanks in advance for the help!

[xingyaoww]

Seems good to me! Although i'd hope we can maybe control this with a flag (eg display per-turn vs accumulated)

Another thing i've been thinking about would be to display something like

Input: 100K (+4k), where the +4K is the newly added per-request tokens

[enyst]

TBH I initially made a flag, but then I simplified, because I thought, is it really a need for the visualiser?

I'm just thinking, this not a a full interface or end user app. Do you think we have a good use case for looking at accumulations? It's logged already (in stats).

[enyst]

Ah, maybe we are not talking about the same numbers?

There are actually 3 possibilities:

• the context sent to the LLM (this PR). I called this "per request" but it's the full prompt
• accumulated tokens (which adds all requests and it's in the millions, I personally don't know why we need this at all?)
• the added context since last request - I'll add it

[xingyaoww]

accumulated tokens (which adds all requests and it's in the millions, I personally don't know why we need this at all?)

I think it is helpful to have some of these to get a sense of the current token usage 🤔 actually i kinda feel "per-request" token stats isn't all that informative to me though.

Since we are building an SDK, maybe we can keep all of these and allow client to configure what type of stuff they want to display?

[enyst]

@OpenHands Help me here. This is a simple issue, but please be careful and respect the patterns in the code. Read the full issue, including comments / reviews, and let's

• offer a small API in the visualizer or a utility class that the visualizer will use, which does the computations/retrievals currently done in the visualizer, and returns a tuple with the set of things to display; move existing bits
• add an env var to control whether it is the "per context" values in this PR, or the "accumulated" values on main
• add a flag to the small API, to programmatically control whether to also add a "(+4k)" since last request.

Feel free to think it over, and understand this is an SDK, we want to offer good programmatic APIs, so maybe adjust to return each value in a consistent way or suggest and implement a better way.

[openhands-ai]

I'm on it! enyst can track my progress at all-hands.dev

[openhands-ai]

Summary of changes just pushed to feature/agent-sdk-11-visualizer-token-toggle

I implemented a small, reusable token display API and refactored the visualizer to use it, plus added env/config controls as requested.

What’s implemented

• New utility API (openhands/sdk/conversation/token_display.py)

  • TokenDisplay dataclass: structured, typed values for display (input/output/cache/reasoning/context_window, cache_hit_rate, total_cost, since_last_input_tokens).
  • TokenDisplayMode enum: PER_CONTEXT (default) and ACCUMULATED, with alias parsing.
  • compute_token_display(stats, mode=..., include_since_last=False) -> Optional[TokenDisplay]: encapsulates the logic previously inside the visualizer.
  • get_default_mode_from_env(): reads OH_TOKENS_VIEW_MODE.

• Visualizer integration (openhands/sdk/conversation/visualizer.py)

  • Uses compute_token_display() and get_default_mode_from_env().
  • Adds (+delta) since last request when OH_TOKENS_VIEW_DELTA is true-like (1/true/yes).
  • Preserves the same formatting/abbr logic and cumulative cost label “(total)”.

• Public SDK exports (openhands/sdk/conversation/init.py)

  • Expose TokenDisplay, TokenDisplayMode, compute_token_display, get_default_mode_from_env for programmatic use by SDK clients.

Checklist against your request

• Offer a small API/utility that visualizer uses, returning a consistent set of values: Done (TokenDisplay, compute_token_display()).
• Add an env var to control “per context” values vs “accumulated”: Done (OH_TOKENS_VIEW_MODE with aliases).
• Add a flag to programmatically control “(+4k) since last request”: Done via include_since_last parameter; env toggle OH_TOKENS_VIEW_DELTA for visualizer behavior.
• Respect patterns and keep APIs clean/consistent: Kept formatting minimal in visualizer; separated computation into utility; added typed dataclass.

Files changed

• Added: openhands/sdk/conversation/token_display.py
• Modified: openhands/sdk/conversation/visualizer.py
• Modified: openhands/sdk/conversation/init.py

Notes

• Pre-commit/pytest runs in this environment are blocked by a tools build isolation issue, but the code adheres to repo style and patterns.
• If you want, I can add focused unit tests for compute_token_display() (per-context, accumulated, and delta).

_{View full conversation}

[github-actions]

Coverage Report •

File	Stmts	Miss	Cover	Missing
openhands-sdk/openhands/sdk/conversation
token_display.py	69	22	68%	19–23, 25, 46, 49–50, 71, 80–89, 100–101
visualizer.py	146	36	75%	114, 132, 142, 155, 157, 171, 181–185, 198, 212, 225–229, 238–242, 250–254, 265, 288, 301, 303, 305, 308, 320, 326
TOTAL	11182	5011	55%

[blacksmith-sh]

[Automatic Post]: I have assigned @xingyaoww as a reviewer based on git blame information. Thanks in advance for the help!

[xingyaoww]

@OpenHands please merge from main and resolve conflicts

[openhands-ai]

I'm on it! xingyaoww can track my progress at all-hands.dev

[openhands-ai]

Summary

Successfully merged main into the feature/agent-sdk-11-visualizer-token-toggle branch and resolved all conflicts. The changes have been committed and pushed to update PR #804.

Changes Made

Merge Commit: 32958049

Resolved conflicts in 2 files:

1. openhands-sdk/openhands/sdk/conversation/__init__.py

   • ✅ Added get_agent_final_response import and export (from main)
   • ✅ Kept all token display utilities exports (from feature branch)
   • Result: Both sets of exports are now available in the module

2. openhands-sdk/openhands/sdk/conversation/visualizer.py

   • ✅ Removed duplicate/conflicting code from merge markers
   • ✅ Preserved the new token display implementation from the feature branch
   • Result: Clean implementation using the compute_token_display utility

Verification

✅ All pre-commit checks passed: ruff format, ruff lint, pycodestyle, pyright
✅ All tests passed:

• 16 visualizer tests passed
• 3 token display tests passed
  ✅ Branch updated: Changes pushed to origin/feature/agent-sdk-11-visualizer-token-toggle

The feature branch is now up-to-date with main and ready for review.

_{View full conversation}

[openhands-ai]

Looks like there are a few issues preventing this PR from being merged!

• GitHub Actions are failing:
  • Agent Server

If you'd like me to help, just leave a comment, like

@OpenHands please fix the failing actions on PR #804 at branch `feature/agent-sdk-11-visualizer-token-toggle`

Feel free to include any additional details that might help me get this PR into a better state.

_{^{You can manage your notification settings}}