feat(insights): make the page actionable — verdict banner, context, run stats - #635
feat(insights): make the page actionable — verdict banner, context, run stats#635mthines wants to merge 9 commits into
Conversation
…tionable Three concrete UX gaps on the flagged /insights page, addressed with data already fetched (no new endpoint): - Friction rows named only a tool+outcome+count with nowhere to go from there. failuresByToolOutcome now also reports the dominant (client, scope_type) context behind each failure, so "187 memory.read errors" reads as "mostly cli, branch (92%)" — a concrete place to go look. - AgentBreakdown's "unattributed" client row had no explanation, unlike the identical bucket in ScopeConsumption. Added the same tooltip explaining why it exists and that it isn't actionable. - ScopeConsumption's "+N more scopes" was inert label text with no way to actually see those scopes. It's now a toggle that expands the full list (scrollable past a point) and collapses back. Regenerated the three touched visual baselines and added an interaction test for the new expand/collapse behavior, which had no prior coverage.
…ats inline Answers the page's core "so what" gap: HealthSummary reads the same /usage fetch's pre-rolled summary + the existing top failure to render one healthy/degraded/unhealthy verdict above the diagnostic panels, instead of making every reader parse Friction/Latency/Coverage gaps to find out whether anything is wrong. Also surfaces each run's read/record/write/scope counts in the collapsed RunsList row (previously visible only after expanding), and reframes the Runs section's description as an audit trail rather than a health signal.
R2 Hoist Shared Constant: export TREND_WINDOW_DAYS from lib/queries/dashboard.ts and have HealthSummary reference it instead of hardcoding "62 days" twice — the duplicate prose literal could silently drift from the real fetch window if it ever changes. Applied via polish (code-quality simplify mode), confidence 97%, scoped typecheck + lint clean, full web test suite green (95 files / 1351 tests).
|
The dashboard preview for this PR — redeployed on each push that changes the web app.
Warning The preview deploy for The Vercel deployment did not complete. See the workflow logs for the CLI output. A blocked deployment is most often the commit author's email not matching a GitHub account with project access. Fix the cause, then push again or comment |
|
Reviewed your changes — no blocking issues, 0 findings. All 4 prior findings confirmed fixed; description now covers the review-pass work too. Review detailsReviewed for commit
CI — All required checks green (Typecheck/Test/Lint, Storybook interaction+visual); Vercel preview deploy still pending (expected on a draft). Run mode — full · 766 lines in delta Memories — 323 indexed · 4 used
Quality — produced 0 → posted inline 0, cleared 0 Integrations — not activated Optimality (2.4c) — ran — reviewed the two-dimensional verdict design and leaderboard click-throughs; no more-optimal alternative identified, 0 proposals Standards (2.4d) — ran — packages/web/CLAUDE.md design-token + REST-only rules re-checked against the delta; 0 violations Skipped files — none (14 screenshot PNGs reviewed as binary diffs only) Reviewed by the |
![dash0-dev[bot]](https://avatars.githubusercontent.com/in/2731631?s=60&v=4){kind=small}
InsightsPage gated the HealthSummary verdict banner on usageByTool.length > 0, so HealthSummary's own tested "No calls recorded" branch (totalCalls === 0) could never render in production — a truly-empty account got no banner at all, unlike every other section on the page. Drop the redundant length guard; HealthSummary already owns isLoading/isError/data-gated rendering via its own totalCalls check. Also fixes the Storybook Empty story, which relied on the shared MSW /usage fixture's nonzero summary.total_events (128) despite empty by_tool — previously invisible because HealthSummary never rendered there. Adds a zeroed /usage override so the story's "every section renders its empty state" claim stays true, and regenerates the affected screenshot baseline. Addresses dash0-dev review comment on PR #635 (#635 (comment)). Claude-Session: https://claude.ai/code/session_01VxLF67tz1z63us1ZuG4zfN
The reads/records/writes/scopes summary was nested inside the run row's <button>, so screen readers announced all four stats on every focus/tab alongside the session badge, correlation id, and timestamp already in the button. Add aria-hidden to keep it visible but out of the accessible name — the expanded <dl> already gives screen-reader users a properly labelled equivalent of the same numbers, so nothing is lost, only de-duplicated. Addresses dash0-dev review comment on PR #635 (#635 (comment)). Claude-Session: https://claude.ai/code/session_01VxLF67tz1z63us1ZuG4zfN
…ivity range picker with trend chips Operational health, the verdict banner, and coverage/latency diagnostics on /insights previously mixed in the reader's own page-loads alongside real agent traffic, understating the true agent success rate — the Explorer's read-activity cards already draw this "browsing isn't consumption" line (migration 00054), Insights just never had it. HealthSummary/UsageHealth now filter it out via excludeDashboardReads; AgentBreakdown's "who is reading" keeps the full set since showing that split is its whole point. Also adds a shared, bounded RangePicker (24h/7d/30d/90d) driving HealthSummary/UsageHealth/AgentBreakdown together via one new hook (useInsightsUsage, which also fetches the immediately preceding window), so a reader can compare "is my agent reading better this week than last" via week-over-week trend chips on the verdict banner, reusing the existing TrendChip/pctChange infrastructure. ScopeConsumption keeps its own separate window, as before. No public-facing docs/llms.txt update: /insights isn't described in the public MDX docs (dashboard-only UX refinement, no new MCP tool/REST route/CLI command/config key). Claude-Session: https://claude.ai/code/session_01VxLF67tz1z63us1ZuG4zfN
|
The Not this PR's failure — the actual Vercel deploy/alias succeeded; only the follow-up "post preview URL" step broke. This PR's base predated PR #636 ("Extract web-preview comment renderer to a testable module"), which added Fix: merged Generated by Claude Code |
…ce to avoid name collision Addresses two non-blocking dash0-dev review findings on commit 7ceea5c: raw hex verdict-icon colors instead of the package's --color-* tokens, and a HealthSummary interface sharing its name with the HealthSummary React component that consumes it. Claude-Session: https://claude.ai/code/session_01VxLF67tz1z63us1ZuG4zfN
… to go A persona review of /insights found the page reads as instrumentation rather than as something that changes what a user does next. Six fixes, all on the existing surface: - The verdict weighs COVERAGE as well as reliability and reports the worse of the two, naming which drove it. Success rate alone never moved: the `usage_events.outcome` vocabulary has no "found nothing" state, so an empty-scope read is `ok` and a healthy account read "100% of calls succeeded" every day in the page's first slot, while "agents are asking for lore that isn't there" sat three columns into the section below. Coverage counts record-bearing tools only, and a window with no reads reports an absence rather than a 0.0 that would read as failure. - `healthTrend` needs 20 calls in the previous window, not merely non-zero. A one-call baseline rendered "+99,900%". - Both leaderboards click through into the Explorer — `?scope=` for a scope row, `?scope=&q=` for a lesson. The unattributed row stays inert; it has no scope to narrow to. - Insights inherits Overview's onboarding duties along with its home slot. The flag hid `/overview`, which was the only host of the pending-invite banner, the first-run checklist and the GitHub App teaser — and the only caller of the first-token mint. A flag-on signup landed on an analytics page with no token and no instructions. - `RangePicker` takes a `label`, so the page's two radiogroups stop sharing one accessible name. - The banner and the Agent activity caption say that dashboard browsing is excluded, rather than leaving it implicit. Claude-Session: https://claude.ai/code/session_01VxLF67tz1z63us1ZuG4zfN
|
❌ The dashboard preview for Open the run logs for the failure. Comment |
|
2 participants
Why
/insights(still flagged off) surfaced raw diagnostics with no "so what" — a failure count with nowhere to click, an "unattributed" client with no explanation, a truncated scope list with no way to see the rest, and a Runs ledger nobody could interpret. Separately, the page's health figures mixed in the dashboard's own page-loads as if they were agent traffic, and there was no way to see whether an agent's usage was trending better or worse over time.A follow-up review of the assembled page found the deeper problem: it still read as instrumentation rather than as something that changes what a reader does next. Its headline figure could not move, its two leaderboards were dead ends, and turning the flag on stranded onboarding on a page nothing links to.
What changed
Making the page legible (first pass)
HealthSummaryverdict banner above the diagnostic panels, reusing the already-fetched usage summaryRunsListinstead of only after expanding, and reframe the section as an audit trail rather than a health signalHealthSummary/UsageHealth(excludeDashboardReads) — browsing the Explorer yourself isn't "how your agents are actually using them";AgentBreakdown's "who is reading" keeps the full set since showing that split is its job24h/7d/30d/90d) drivingHealthSummary/UsageHealth/AgentBreakdownfrom one fetch, plus week-over-week trend chips (healthTrend)Making it actionable (review pass)
healthVerdictgrades reliability andreadCoverage's records-per-read and reports the worse of the two, with the banner naming which drove it. Success rate alone never moved:usage_events.outcomeisok | cap_exceeded | rate_limited | permission_denied | error, none of which means "the agent asked for lore and got nothing" — an empty-scope read isok. So a healthy account read "100% of calls succeeded" every day in the page's largest, first slot, while the signal a reader came for sat three columns into the section below. Coverage counts record-bearing tools only (memory.list/read/search), sincememory.write/org.*are structurallyrecord_count: 0; a window with no reads reports an absence ("no reads"), never a0.0that would read as failure.healthTrendneeds a real baseline — 20 calls in the previous window, not merely non-zero. One prior call against a thousand rendered "+99,900%".ScopeConsumptionrow links to/lore?scope=SCOPE, aHotColdLorerow to/lore?scope=SCOPE&q=KEY. Theunattributedrow stays inert — it has no scope to narrow to, and linking it would show a different set of lore than the bar measures. Two encoding constraints:scopeis deliberately not a?filters=dimension, and the Explorer has no?lesson=param (its detail sheet is local state), so scope + search is the closest honest target./overviewfrom nav, and/overviewwas the only host ofPendingInvitesBanner/OnboardingChecklist/GithubAppTeaser— and the only caller ofbuildOnboardingSteps({ autoGenerateToken: true }), which mints a brand-new user's first API token. A flag-on signup landed on an analytics page with no token, no setup instructions, and no way to find either. All four move ontoinsights/page.tsx, the root redirect points at/insights, and Insights takes the first nav slot so the rail matches the claim. All three surfaces self-hide once irrelevant, so an established account sees the page unchanged.RangePickertakes alabel, so the page's two independent radiogroups stop sharing one accessible name.How to verify
pnpm nx typecheck web && pnpm nx lint web && pnpm nx test webcd packages/web && npx vitest run --config vitest.storybook.config.ts --changed=mainVisual baselines for
HealthSummaryandInsightsPagewere regenerated (-u) for the banner's new two-figure layout; two newHealthSummarystories cover the coverage-driven verdict and the write-only window.Generated by Claude Code