Skip to content

Show prompt-cache tokens and cost savings on the Monitor page #10

@jstuart0

Description

@jstuart0

Summary

The agentic loop reports cacheCreationInputTokens and cacheReadInputTokens on every ask's AskDiagnostics (populated since v0.9.0-rc.1 / quality-push Phase 1). Benchmark runs show a 99.6% cache-read ratio and ~70% input-token savings — a real cost win worth making visible.

Today none of those fields surface in the web UI. Operators have to parse AskDiagnostics by hand to see the savings.

What needs to happen

  1. On the Monitor page (admin LLM activity), add two new columns/rows for the agentic rows:
    • cache_read_tokens
    • cache_creation_tokens
  2. Add a derived "cache savings %" = cache_read / (cache_read + cache_create + regular_input).
  3. On the dashboard home page, add a small stat card showing aggregate cache savings across the last 14 UTC days.
  4. No backend changes — the diagnostics already flow through AskDiagnostics.

Where to look

  • web/src/app/(app)/admin/llm/page.tsx — LLM job list (target for the per-job view).
  • web/src/app/(app)/page.tsx — dashboard home (target for the aggregate stat).
  • web/src/lib/graphql/queries.ts — may need to add the new diagnostic fields to the query selection set.
  • internal/qa/types.goAskDiagnostics with CacheCreationInputTokens + CacheReadInputTokens.

Acceptance

  • The LLM activity page shows cache-read and cache-creation token counts per agentic job.
  • A "cache savings" percentage is rendered per-row.
  • The dashboard home shows an aggregate number.

Difficulty

Beginner-friendly for React/TypeScript developers. Similar shape to issue #6 (LLM cost rendering) — data is available, just needs to be displayed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestgood first issueGood for newcomers — well-scoped, isolated, clear acceptance criteria

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions