[Enhancement] Add LM Studio-style token throughput + total token count display in message footer

[markwang2658]

## Summary

Add LM Studio-style real-time token throughput (token/s) and **total token countdisplay in the assistant message footer, showingXX.X token/s · XXXX token` format.

Motivation / Use Case

When using LLM chat interfaces (especially for cost monitoring and performance tuning), users need quick visibility into:

1. Token generation speed — how fast the model is responding
2. Total tokens consumed per turn — for cost tracking and context window awareness

LM Studio already provides this UX pattern and users familiar with it expect similar information density. Currently Hermes WebUI shows input/output token counts but lacks:

• Real-time generation speed (token/s)
• Aggregated total token count in a prominent position

Proposed Behavior

Display Format

┌─────────────────────────────────────────────────────┐
│ [Assistant response content...]                     │
├─────────────────────────────────────────────────────┤
│ 15.0k in · 115 out   50.4 token/s · 15.1k token   │
│                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^  │
│              Speed (TPS) + Total Tokens             │
└─────────────────────────────────────────────────────┘

Two States

State	Display	When
Streaming (in progress)	XX.X token/s only	metering events fire, _turnUsage not yet available
Completed (done)	XX.X token/s · XXXX token	done event fires, _turnUsage populated

Formatting Rules

• Speed first, token count second (LM Studio convention)
• Separator: · (middle dot, consistent with existing in · out format)
• Unit: full word token (not abbreviated tok)
• Large numbers: auto-format with suffixes (15000 → 15.0k, 2500000 → 2.5M)
• Position: inside .msg-foot, after .msg-usage-inline or .msg-time element
• Style: muted color, small font (12px), semi-transparent (opacity 0.85), bold weight

Technical Approach

Data Sources (Already Available)

Data	Source Event	Field
input_tokens	done	_turnUsage.input_tokens
output_tokens	done	_turnUsage.output_tokens
total_tokens	computed	input_tokens + output_tokens
tokens/sec	metering	calculated from token delta / time delta

Implementation Scope

Single file modification: static/messages.js

1. New helper function _fmtTokens(n) — ~7 lines

   • Formats large numbers with k/M suffixes

2. Enhanced function _injectLiveTpsToMessage(tps) — ~15 lines modification

   • Reads last assistant message's _turnUsage from S.messages global array
   • Computes total_tokens = input + output
   • Conditionally renders speed-only or speed+tokens format
   • Creates/updates .msg-speed-inline DOM element

Total change: ~22 lines of code across 2 functions in 1 file.

No Backend Changes Required

All data is already emitted by the backend via WebSocket events (metering, done). This is purely a frontend enhancement.

Reference Implementation

A working proof-of-concept is available at:
🔗 messages.js (fork with enhancement)

Note: This is a fork for demonstration only. The relevant changes are in _fmtTokens() and _injectLiveTpsToMessage() functions (~22 lines total, approximately lines 1199-1240).

Tested on Hermes Agent v0.13.0 (2026.5.7) with the following verified results:

• ✅ Streaming phase shows speed only
• ✅ Completion phase shows XX.X token/s · XXXX token
• ✅ Multi-turn conversations display independently per message
• ✅ No duplicate elements, no layout breakage
• ✅ Works with existing in · out usage display
• ✅ Zero backend changes needed

Key Code Snippet (Reference)

// Helper: format large numbers
function _fmtTokens(n) {
  if (!n || n < 0) return '0';
  if (n >= 1e6) return (n / 1e6).toFixed(1) + 'M';
  if (n >= 1e3) return (n / 1e3).toFixed(1) + 'k';
  return String(n);
}

// Enhanced injection (core logic)
function _injectLiveTpsToMessage(tps) {
  const activeFoot = document.querySelector('.assistant-turn:last-child .msg-foot');
  if (!activeFoot) return;

  // Get usage from global S.messages array (reliable data source)
  const lastAsstMsg = S.messages && Array.isArray(S.messages)
    ? [...S.messages].reverse().find(m => m.role === 'assistant') : null;
  const usage = lastAsstMsg?._turnUsage || null;
  const totalTok = (usage?.input_tokens || 0) + (usage?.output_tokens || 0);

  const tpsVal = typeof tps === 'number' ? tps.toFixed(1) : '—';
  // ... DOM element creation/update ...
  
  // Render: speed + token (LM Studio style)
  speedEl.textContent = totalTok > 0
    ? `${tpsVal} token/s · ${_fmtTokens(totalTok)} token`
    : `${tpsVal} token/s`;
}

Alternatives Considered

Option	Pros	Cons	Verdict
A: Current proposal (DOM injection in messages.js)	Minimal change, zero backend impact	Not persisted on page refresh	✅ Selected
B: Modify ui.js renderMessages()	Persisted on refresh	Higher risk of breaking changes, larger diff	❌ Rejected
C: Backend-side footer enhancement	Always available	Requires server code change, more complex	❌ Overkill
D: Browser extension	No source change needed	Maintenance burden, distribution problem	❌ Impractical

Known Limitations

1. Non-persistent on refresh: Speed/token info disappears on page reload (same behavior as current TPS display). Fixing this would require modifying ui.js render logic.
2. Requires metering events: If streaming doesn't emit metering events, speed won't show (token count from done event still works).

Screenshots / Mockup

During streaming:

15.0k in · 115 out   50.4 token/s    10:42 AM
                        ↑
                   Speed only (usage not yet available)

After completion:

15.0k in · 115 out   50.4 token/s · 15.1k token    10:43 AM
                       ^^^^^^^^^^^^^^^^^^^^^^^^
                  Speed + Total Token Count

Checklist for Implementation

• Add _fmtTokens() helper function to messages.js
• Enhance _injectLiveTpsToMessage() with token count logic
• Test streaming state (speed-only display)
• Test completion state (speed + token display)
• Test multi-turn conversation isolation
• Verify no regression in existing usage display
• Test on Chrome, Firefox, Edge

Labels Suggested

enhancement ui good first issue webui

---

Note: This is a ready-to-implement feature request with a working reference implementation. The approach requires modifying only messages.js (~22 lines) with no backend changes.

---

[nesquena-hermes]

Summary

Read the proposal end-to-end against the existing TPS implementation in static/ui.js:4148-4170 and the metering wire-up in static/messages.js:1158-1167. The infrastructure for the speed half of this proposal already exists — there's a .msg-tps-inline chip that gates on window._showTps===true, fed by the metering SSE event. The token-count half is the new ask.

Code reference

The metering pipeline is already in place at static/messages.js:1158-1167:

source.addEventListener('metering',e=>{
  try{
    const d=JSON.parse(e.data||'{}');
    if((d.session_id||activeSid)!==activeSid) return;
    if(d.estimated===true||d.tps_available!==true||typeof d.tps!=='number'||d.tps<=0){
      if(typeof _setLiveAssistantTps==='function') _setLiveAssistantTps(null);
      return;
    }
    if(typeof _setLiveAssistantTps==='function') _setLiveAssistantTps(d.tps);
  }catch(_){}
});

_setLiveAssistantTps() at static/ui.js:4168-4170 writes into a .msg-tps-inline chip rendered inside .msg-role, gated on isTpsDisplayEnabled() === window._showTps===true. The token-count side already lives in .msg-usage-inline at static/ui.js:5263-5273, gated on window._showTokenUsage:

if(window._showTokenUsage&&hasTurnUsage){
  const usage=document.createElement('span');
  usage.className='msg-usage-inline';
  const inTok=msg._turnUsage.input_tokens||0;
  const outTok=msg._turnUsage.output_tokens||0;
  const cost=msg._turnUsage.estimated_cost;
  let text=`${_fmtTokens(inTok)} in · ${_fmtTokens(outTok)} out`;
  ...
}

_fmtTokens() at static/ui.js:1732 already does the k/M suffix formatting your proposal would re-implement.

Diagnosis

So the request is really three deltas, not the 22-line greenfield it reads as:

1. Move the TPS chip: today it lives in .msg-role (top of the assistant turn). Your mockup wants it in .msg-foot next to .msg-usage-inline. That's a re-target of _setAssistantTurnTps() at static/ui.js:4153-4167.

2. Append total tokens to the speed chip post-completion: when done fires and _turnUsage populates, the chip text becomes ${tps} token/s · ${_fmtTokens(in+out)} token. This is genuinely additive — would live alongside the existing .msg-usage-inline "in / out" split.

3. Persist on render: today .msg-tps-inline gets cleared on renderMessages() because it's only injected during live streaming via _setLiveAssistantTps(). To survive page reload, you'd need to read msg._turnUsage.tps (or whatever field carries final TPS) inside the if(window._showTokenUsage&&hasTurnUsage) block at static/ui.js:5263 and render a sibling chip alongside .msg-usage-inline.

Recommendation

Two things to confirm before this is straightforward to merge:

A. The redundancy question. Today users who flip on _showTokenUsage already see 15.0k in · 115 out in the foot. Your proposal adds 15.1k token (their sum) right next to it. That's redundant for users who already enabled token usage, but useful for users who only enable TPS and don't want the in/out split. If the position of the LM-Studio-style XX.X token/s · XXXX token is purely replacement for the XXXX in · YYY out chip when only TPS is on, that's cleaner — but then it's actually a settings interaction, not just messages.js work.

B. The persistence question. The "speed" half of the chip can't survive reload (TPS is a live measurement; after-the-fact it's output_tokens / response_duration — derivable from _turnUsage if the backend stamps a duration on the turn, but not from _turnUsage.tps because that field doesn't exist today). You'd need either (i) the backend to stamp final TPS into _turnUsage at done-time, or (ii) accept that reloaded sessions show only the token-count half, no speed.

If you're up for opening a PR, the lowest-friction shape would be:

• One settings flag: _showTpsAndTokens (or extend _showTps). Off by default.
• When on: render speed-only chip during streaming via existing _setLiveAssistantTps, then on done extend the chip to ${tps} token/s · ${_fmtTokens(total)} token.
• For persistence: backend stamps _turnUsage.tps and _turnUsage.duration_ms on done. Frontend reads them in the foot-render block at static/ui.js:5237.

Worth noting: there's an existing .msg-tps-inline selector with CSS at static/style.css:2876 and the chip is gated on isTpsDisplayEnabled(). Reuse that path rather than introducing .msg-speed-inline — fewer styles, fewer settings, less surface to keep coherent.

Settings-wise the current toggles (_showTps, _showTokenUsage, _showTokenUsage-derived cost) are already a small matrix. Adding a fourth state worth thinking about; consolidating _showTps and the new behavior into one "live performance" toggle with finer per-element control would be cleaner than another flag.

Happy to review a contributor PR once these two design questions are settled. Mark me as reviewer when ready.

[nesquena-hermes]

Closing as completed through the current per-turn metadata surface.

The implementation differs slightly from the LM Studio mockup, but the requested information now appears on each assistant turn:

• TPS is rendered as an opt-in t/s chip beside the assistant name, using the backend's measured per-turn usage.tps rather than a browser estimate;
• settled Transparent Stream turns show input and output token counts in their footer (N in · M out), alongside duration, time-to-first-token, and final status;
• session-level cumulative input/output/cache/cost data remains available in the context/usage indicator;
• ephemeral TPS/usage metadata is carried forward across force reloads and transcript refreshes so the badges do not disappear.

Verification during closure triage:

9 passed in 3.91s

The targeted TPS suite pins header placement, live updates, disabled-state cleanup, responsive layout, and full-reload carry-forward. The Transparent Stream footer suite had already passed in the same triage run and pins the token footer.

This supplies the throughput and token-count diagnostics requested here without making telemetry mandatory for users who prefer a quieter transcript. Thanks @labovnix for the concrete LM Studio reference and placement idea.