feat(dashboard): show gateway and provider cache on Usage page#336
Merged
Conversation
Show cached usage entries by default on the Usage page with an opt-in
"Hide cached requests" checkbox that filters both the paginated log and
the live SSE stream. The broker now broadcasts cached usage events so
the dashboard can choose whether to surface them. Cached rows are
visually distinguished (italic, dimmed via opacity), include a Cache
column ("Exact"/"Semantic" in bold), and a database-zap icon next to
the cost with a "Saved by cache — not charged" tooltip on the ancestor
cell.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add a Provider Cache column to the Usage page request log showing the share of input tokens served from the upstream provider's prompt cache (OpenAI prefix cache, Anthropic ephemeral cache, DeepSeek context cache, Gemini cache). Reorder columns to Provider | Model. Keep cached-row formatting (italic + nowrap for cost cells so the cache icon stays inline with the price), and widen the dashboard content area to 1400px. Refactor: extract usage.EntryInputSegments so SummarizeRequestUsage, the admin usage log handler, and the live SSE preview share one provider-cache segmenter (handles Anthropic's split accounting where input_tokens is the uncached portion and cache_read/creation are reported alongside). The derived fields (uncached_input_tokens, cached_input_tokens, cache_write_input_tokens, cached_input_ratio) are JSON-only on UsageLogEntry, populated by EnrichUsageLogEntry at the admin/SSE boundary so storage layers stay untouched. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
📝 WalkthroughWalkthroughAdds server-side cache-derived fields and EnrichUsageLogEntry, broadcasts enriched usage events (including cached), frontend toggle ChangesCache-aware usage log filtering and enrichment
Sequence DiagramsequenceDiagram
participant DashboardClient
participant DashboardJS
participant HandlerUsage
participant EnrichUsageLogEntry
DashboardClient->>DashboardJS: fetchUsageLog(cache_mode)
DashboardJS->>HandlerUsage: GET /admin/usage/log?cache_mode=...
HandlerUsage->>EnrichUsageLogEntry: EnrichUsageLogEntry(&entry) for each entry
EnrichUsageLogEntry->>HandlerUsage: return enriched entry
HandlerUsage->>DashboardJS: return enriched JSON entries
sequenceDiagram
participant Broker
participant EnrichUsageLogEntry
participant Publisher
participant DashboardLiveLogs
Broker->>EnrichUsageLogEntry: usagePreviewFromEntry(entry)
EnrichUsageLogEntry->>Broker: enriched preview
Broker->>Publisher: publish enriched EventUsageCompleted
Publisher->>DashboardLiveLogs: live event received
DashboardLiveLogs->>DashboardLiveLogs: skip insert if usageLogHideCached && entry.cached
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested labels
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Greptile SummaryThis PR expands the Usage page to show gateway and provider cache details. It changes:
Confidence Score: 5/5This looks safe to merge.
Important Files Changed
Reviews (2): Last reviewed commit: "test(usage): pin max-coalescing of provi..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
EntryInputSegments only read cache_creation_input_tokens (Anthropic's field name) for cache writes; Bedrock's Converse extras emit cache_write_input_tokens (matching AWS's field name). Cache writes from Bedrock were silently zero on the Usage page, the live SSE preview, and in request summaries, skewing the provider-cache ratio against the wrong total. Coalesce both keys via max — consistent with how multiple cached-read field names are merged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@internal/admin/dashboard/static/js/modules/live-logs.js`:
- Around line 290-292: The guard using this.usageLogHideCached and
this.liveUsageEntryCached(incoming) only returns for new items (index < 0) but
doesn't remove or stop updates to existing cached rows; update the logic in the
handler that processes incoming and index so that when this.usageLogHideCached
is true and this.liveUsageEntryCached(incoming) is true you also evict any
existing row (when index >= 0) instead of leaving it visible—e.g., locate the
insert/update branch that references index and incoming and either remove the
existing DOM row or skip updating it when the item is cached and hide-cached is
enabled, ensuring the same check (this.usageLogHideCached &&
this.liveUsageEntryCached(incoming)) applies to both insert and update paths.
In `@internal/admin/dashboard/static/js/modules/usage.js`:
- Around line 367-370: The cached-entry detection in usageEntryCached currently
returns true only for 'exact' and 'semantic' but must also treat 'cache_hit' as
cached to match live-log semantics; update the usageEntryCached function to
include 'cache_hit' (or normalize 'cache_hit' to the same branch as 'semantic')
when calling usageEntryCacheType so fetched rows get the same cached
styling/tooltip as live rows.
In `@internal/admin/dashboard/templates/page-usage.html`:
- Around line 157-159: Add an explicit id/for association for the "Hide cached
requests" checkbox: give the input element a stable id (e.g.
id="usage-log-hide-cached") and change the label to target that id via
for="usage-log-hide-cached" while keeping the existing class
"usage-log-checkbox", x-model="usageLogHideCached", and
`@change`="fetchUsageLog(true)"; ensure the input is not nested inside the label
so the label/for pair satisfies HTMLHint's input-requires-label rule.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 94cff39a-bb2a-433f-baae-8317eb5f8f0f
📒 Files selected for processing (14)
internal/admin/dashboard/static/css/dashboard.cssinternal/admin/dashboard/static/js/dashboard.jsinternal/admin/dashboard/static/js/modules/dashboard-layout.test.cjsinternal/admin/dashboard/static/js/modules/live-logs.jsinternal/admin/dashboard/static/js/modules/live-logs.test.cjsinternal/admin/dashboard/static/js/modules/usage.jsinternal/admin/dashboard/static/js/modules/usage.test.cjsinternal/admin/dashboard/templates/page-usage.htmlinternal/admin/handler_usage.gointernal/live/broker.gointernal/live/broker_test.gointernal/usage/enrich_test.gointernal/usage/reader.gointernal/usage/request_summary.go
Comment on lines
+290
to
+292
| if (this.usageLogHideCached && this.liveUsageEntryCached(incoming) && index < 0) { | ||
| return; | ||
| } |
Contributor
There was a problem hiding this comment.
Hide-cached filtering should also evict existing cached rows.
On Line 290, the guard only skips inserts (index < 0). If a cached row already exists, it can remain visible/updated even while hide-cached is enabled.
Suggested fix
- if (this.usageLogHideCached && this.liveUsageEntryCached(incoming) && index < 0) {
- return;
- }
+ if (this.usageLogHideCached && this.liveUsageEntryCached(incoming)) {
+ if (index >= 0) {
+ currentEntries.splice(index, 1);
+ this.usageLog.entries = [...currentEntries];
+ }
+ return;
+ }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@internal/admin/dashboard/static/js/modules/live-logs.js` around lines 290 -
292, The guard using this.usageLogHideCached and
this.liveUsageEntryCached(incoming) only returns for new items (index < 0) but
doesn't remove or stop updates to existing cached rows; update the logic in the
handler that processes incoming and index so that when this.usageLogHideCached
is true and this.liveUsageEntryCached(incoming) is true you also evict any
existing row (when index >= 0) instead of leaving it visible—e.g., locate the
insert/update branch that references index and incoming and either remove the
existing DOM row or skip updating it when the item is cached and hide-cached is
enabled, ensuring the same check (this.usageLogHideCached &&
this.liveUsageEntryCached(incoming)) applies to both insert and update paths.
Comment on lines
+367
to
+370
| usageEntryCached(entry) { | ||
| const type = this.usageEntryCacheType(entry); | ||
| return type === 'exact' || type === 'semantic'; | ||
| }, |
Contributor
There was a problem hiding this comment.
Unify cached-entry detection with live-log semantics.
Line 367 currently treats only exact|semantic as cached, while live-log handling also treats cache_hit as cached. This can cause inconsistent cache styling/tooltip behavior between fetched rows and live rows.
Suggested fix
usageEntryCached(entry) {
const type = this.usageEntryCacheType(entry);
- return type === 'exact' || type === 'semantic';
+ return type === 'exact' || type === 'semantic' || !!(entry && entry.cache_hit);
},📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| usageEntryCached(entry) { | |
| const type = this.usageEntryCacheType(entry); | |
| return type === 'exact' || type === 'semantic'; | |
| }, | |
| usageEntryCached(entry) { | |
| const type = this.usageEntryCacheType(entry); | |
| return type === 'exact' || type === 'semantic' || !!(entry && entry.cache_hit); | |
| }, |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@internal/admin/dashboard/static/js/modules/usage.js` around lines 367 - 370,
The cached-entry detection in usageEntryCached currently returns true only for
'exact' and 'semantic' but must also treat 'cache_hit' as cached to match
live-log semantics; update the usageEntryCached function to include 'cache_hit'
(or normalize 'cache_hit' to the same branch as 'semantic') when calling
usageEntryCacheType so fetched rows get the same cached styling/tooltip as live
rows.
Comment on lines
+157
to
+159
| <label class="usage-log-checkbox"> | ||
| <input type="checkbox" x-model="usageLogHideCached" @change="fetchUsageLog(true)"> | ||
| <span>Hide cached requests</span> |
Contributor
There was a problem hiding this comment.
Add explicit for/id association for the hide-cached checkbox.
Line 158 is currently flagged by HTMLHint (input-requires-label). An explicit for/id pair will satisfy lint and keep accessibility intent unambiguous.
Suggested fix
- <label class="usage-log-checkbox">
- <input type="checkbox" x-model="usageLogHideCached" `@change`="fetchUsageLog(true)">
+ <label class="usage-log-checkbox" for="usage-hide-cached">
+ <input id="usage-hide-cached" type="checkbox" x-model="usageLogHideCached" `@change`="fetchUsageLog(true)">
<span>Hide cached requests</span>
</label>📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| <label class="usage-log-checkbox"> | |
| <input type="checkbox" x-model="usageLogHideCached" @change="fetchUsageLog(true)"> | |
| <span>Hide cached requests</span> | |
| <label class="usage-log-checkbox" for="usage-hide-cached"> | |
| <input id="usage-hide-cached" type="checkbox" x-model="usageLogHideCached" `@change`="fetchUsageLog(true)"> | |
| <span>Hide cached requests</span> | |
| </label> |
🧰 Tools
🪛 HTMLHint (1.9.2)
[warning] 158-158: No matching [ label ] tag found.
(input-requires-label)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@internal/admin/dashboard/templates/page-usage.html` around lines 157 - 159,
Add an explicit id/for association for the "Hide cached requests" checkbox: give
the input element a stable id (e.g. id="usage-log-hide-cached") and change the
label to target that id via for="usage-log-hide-cached" while keeping the
existing class "usage-log-checkbox", x-model="usageLogHideCached", and
`@change`="fetchUsageLog(true)"; ensure the input is not nested inside the label
so the label/for pair satisfies HTMLHint's input-requires-label rule.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@internal/usage/enrich_test.go`:
- Around line 74-118: Add a test that verifies when both bedrock
cache_creation_input_tokens and cache_write_input_tokens are present the code
picks the larger value; create or extend a test (e.g.,
TestEnrichUsageLogEntry_BedrockCacheWriteMax) that constructs a UsageLogEntry
with Provider "bedrock", InputTokens set, and RawData containing both
"cache_creation_input_tokens" and "cache_write_input_tokens" with different
values, call EnrichUsageLogEntry(&entry), and assert entry.CacheWriteInputTokens
equals the max of the two values (and keep existing assertions for
UncachedInputTokens/CachedInputTokens as appropriate) to lock in max-coalescing
behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 4b3ec8ac-7732-442a-9614-1ddb5ed789af
📒 Files selected for processing (2)
internal/usage/enrich_test.gointernal/usage/request_summary.go
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Add a regression test for the case where raw_data carries both cache_creation_input_tokens and cache_write_input_tokens; the segmenter must pick the larger value so provider-cache write totals aren't undercounted on the dashboard. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@internal/usage/enrich_test.go`:
- Around line 57-72: Add a new unit test alongside
TestEnrichUsageLogEntry_NoCacheData that constructs a UsageLogEntry with
Provider "openai" and InputTokens set to 0, calls EnrichUsageLogEntry(&entry),
and asserts that UncachedInputTokens and CachedInputTokens are zero and that
CachedInputRatio is exactly 0; reference the existing test name
TestEnrichUsageLogEntry_NoCacheData and the EnrichUsageLogEntry function and the
UsageLogEntry fields InputTokens and CachedInputRatio to locate where to add the
case.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 9ef84968-e134-4be7-b93a-5735865e376f
📒 Files selected for processing (1)
internal/usage/enrich_test.go
Comment on lines
+57
to
+72
| func TestEnrichUsageLogEntry_NoCacheData(t *testing.T) { | ||
| entry := UsageLogEntry{ | ||
| Provider: "openai", | ||
| InputTokens: 100, | ||
| } | ||
| EnrichUsageLogEntry(&entry) | ||
| if entry.UncachedInputTokens != 100 { | ||
| t.Fatalf("UncachedInputTokens = %d, want 100", entry.UncachedInputTokens) | ||
| } | ||
| if entry.CachedInputTokens != 0 { | ||
| t.Fatalf("CachedInputTokens = %d, want 0", entry.CachedInputTokens) | ||
| } | ||
| if entry.CachedInputRatio != 0 { | ||
| t.Fatalf("CachedInputRatio = %f, want 0", entry.CachedInputRatio) | ||
| } | ||
| } |
Contributor
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Check EnrichUsageLogEntry and EntryInputSegments for division-by-zero handling
# Search for ratio calculation in the implementation
rg -n -A5 -B5 'CachedInputRatio.*=' internal/usage/Repository: ENTERPILOT/GoModel
Length of output: 4632
🏁 Script executed:
# Find the EnrichUsageLogEntry function definition
rg -n 'func EnrichUsageLogEntry' internal/usage/
# Also check the full enrich.go file to understand the implementation
wc -l internal/usage/enrich.goRepository: ENTERPILOT/GoModel
Length of output: 214
🏁 Script executed:
cat internal/usage/enrich_test.goRepository: ENTERPILOT/GoModel
Length of output: 4308
Add test for InputTokens=0 edge case to ensure explicit coverage.
The test suite lacks coverage for the case where InputTokens is zero. Although the implementation already safely handles this scenario with a guard condition, adding an explicit test locks in the expected behavior. Include a test that verifies CachedInputRatio is correctly set to 0 when InputTokens is 0.
Proposed test addition
func TestEnrichUsageLogEntry_NoCacheData(t *testing.T) {
entry := UsageLogEntry{
Provider: "openai",
InputTokens: 100,
}
EnrichUsageLogEntry(&entry)
if entry.UncachedInputTokens != 100 {
t.Fatalf("UncachedInputTokens = %d, want 100", entry.UncachedInputTokens)
}
if entry.CachedInputTokens != 0 {
t.Fatalf("CachedInputTokens = %d, want 0", entry.CachedInputTokens)
}
if entry.CachedInputRatio != 0 {
t.Fatalf("CachedInputRatio = %f, want 0", entry.CachedInputRatio)
}
}
+func TestEnrichUsageLogEntry_ZeroInputTokens(t *testing.T) {
+ entry := UsageLogEntry{
+ Provider: "openai",
+ InputTokens: 0,
+ }
+ EnrichUsageLogEntry(&entry)
+ if entry.CachedInputRatio != 0 {
+ t.Fatalf("CachedInputRatio = %f, want 0 when InputTokens is 0", entry.CachedInputRatio)
+ }
+}
+
func TestEnrichUsageLogEntry_BedrockCacheWriteField(t *testing.T) {📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| func TestEnrichUsageLogEntry_NoCacheData(t *testing.T) { | |
| entry := UsageLogEntry{ | |
| Provider: "openai", | |
| InputTokens: 100, | |
| } | |
| EnrichUsageLogEntry(&entry) | |
| if entry.UncachedInputTokens != 100 { | |
| t.Fatalf("UncachedInputTokens = %d, want 100", entry.UncachedInputTokens) | |
| } | |
| if entry.CachedInputTokens != 0 { | |
| t.Fatalf("CachedInputTokens = %d, want 0", entry.CachedInputTokens) | |
| } | |
| if entry.CachedInputRatio != 0 { | |
| t.Fatalf("CachedInputRatio = %f, want 0", entry.CachedInputRatio) | |
| } | |
| } | |
| func TestEnrichUsageLogEntry_NoCacheData(t *testing.T) { | |
| entry := UsageLogEntry{ | |
| Provider: "openai", | |
| InputTokens: 100, | |
| } | |
| EnrichUsageLogEntry(&entry) | |
| if entry.UncachedInputTokens != 100 { | |
| t.Fatalf("UncachedInputTokens = %d, want 100", entry.UncachedInputTokens) | |
| } | |
| if entry.CachedInputTokens != 0 { | |
| t.Fatalf("CachedInputTokens = %d, want 0", entry.CachedInputTokens) | |
| } | |
| if entry.CachedInputRatio != 0 { | |
| t.Fatalf("CachedInputRatio = %f, want 0", entry.CachedInputRatio) | |
| } | |
| } | |
| func TestEnrichUsageLogEntry_ZeroInputTokens(t *testing.T) { | |
| entry := UsageLogEntry{ | |
| Provider: "openai", | |
| InputTokens: 0, | |
| } | |
| EnrichUsageLogEntry(&entry) | |
| if entry.CachedInputRatio != 0 { | |
| t.Fatalf("CachedInputRatio = %f, want 0 when InputTokens is 0", entry.CachedInputRatio) | |
| } | |
| } | |
| func TestEnrichUsageLogEntry_BedrockCacheWriteField(t *testing.T) { |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@internal/usage/enrich_test.go` around lines 57 - 72, Add a new unit test
alongside TestEnrichUsageLogEntry_NoCacheData that constructs a UsageLogEntry
with Provider "openai" and InputTokens set to 0, calls
EnrichUsageLogEntry(&entry), and asserts that UncachedInputTokens and
CachedInputTokens are zero and that CachedInputRatio is exactly 0; reference the
existing test name TestEnrichUsageLogEntry_NoCacheData and the
EnrichUsageLogEntry function and the UsageLogEntry fields InputTokens and
CachedInputRatio to locate where to add the case.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
white-space: nowrapso trailing icons stay inline.usage.EntryInputSegmentssoSummarizeRequestUsage, the admin usage log handler, and the live SSE preview share one provider-cache segmenter — handles Anthropic's split accounting whereinput_tokensis the uncached portion andcache_read/creation_input_tokensare reported alongside..contentfrom 1200px to 1400px.Derived fields (
uncached_input_tokens,cached_input_tokens,cache_write_input_tokens,cached_input_ratio) are JSON-only onUsageLogEntry, populated byEnrichUsageLogEntryat the admin/SSE boundary so storage layers stay untouched.Test plan
go test ./...(full Go suite)node --test internal/admin/dashboard/static/js/modules/*.test.cjs(292 tests)🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Improvements
Tests