Skip to content

dashboard(results): adapt target_error_kind reader to slim index rows - #1701

Merged
christso merged 1 commit into
mainfrom
cpl5-dashboard
Jul 6, 2026
Merged

dashboard(results): adapt target_error_kind reader to slim index rows#1701
christso merged 1 commit into
mainfrom
cpl5-dashboard

Conversation

@christso

@christso christso commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

Implements av-cpl5.1 (leaf of parent tracker av-cpl5): adapt Dashboard/CLI result readers to the slim summary.json/index.jsonl shapes shipped in a983924e (cases→tests rename) and 16a21b33 (target_execution/transcript_summary → sidecars).

Investigation found most readers (manifest.ts, export.ts, projection-bundle.ts, summary.ts, report.ts) never referenced the removed cases[]/total_cases/total_instances summary fields or the row-level target_execution/transcript_summary fields directly, so no changes were needed there. summaryRunId() in core already has the run_id root/metadata fallback. The trial-level (samples[]) transcript_summary sidecar fallback in serve.ts's buildRepeatTrialReadModels already covered the new schema (reads result.json when absent inline).

The one real gap: Dashboard's ResultTable.tsx targetErrorKind() helper only read the old nested target_execution.error_kind/errorKind shape, and serve.ts's row read-model never actually passed a row-level target-error signal through to the client at all (for non-repeat rows) — so this was silently broken even before the schema change.

  • apps/cli/src/commands/results/manifest.ts: type target_error_kind (new) and legacy target_execution (fat-row fallback) on ResultManifestRecord.
  • apps/cli/src/commands/results/serve.ts: add recordTargetErrorKind() compat adapter and wire it into attachRunDetailReadModelFields so results[].target_error_kind is populated from either the new compact field or the legacy nested envelope.
  • apps/dashboard/src/lib/types.ts: add target_error_kind to EvalCaseTrial/EvalResult.
  • apps/dashboard/src/components/ResultTable.tsx: targetErrorKind() now prefers target_error_kind, falling back to the legacy nested shape.
  • Tests: new/legacy coverage in serve.test.ts (API-level) and ResultTable.test.tsx (render-level).

Verification

  • bun run build && bun run typecheck — clean.
  • node_modules/.bin/biome check on touched files — clean.
  • Focused tests: apps/cli/test/commands/results/** (249 pass), apps/dashboard/src/lib/result-table.test.ts + ResultTable.test.tsx + transcript-timeline.test.tsx (26 pass).
  • Live dogfood: ran a real eval (examples/features/rubric/evals/operators.eval.yaml) against a live Azure OpenAI target with a live llm-rubric grader. Confirmed the produced summary.json has the slim shape (tests[], total_tests/passed_tests/failed_tests, no metadata.run_id) and .internal/index.jsonl rows have no target_execution/transcript_summary. Verified agentv results show/summary/report all work against the bundle.
  • Green (Dashboard, new slim shape): started agentv dashboard, confirmed /api/runs and /api/runs/:id render the live bundle correctly.
  • Green (Dashboard, legacy compat): hand-built a synthetic legacy fat-shape bundle (cases[] summary + row-level target_execution.error_kind) and confirmed /api/runs/:id correctly derives target_error_kind: "timeout" via the new compat fallback.
  • Red (pre-fix): before the serve.ts/ResultTable.tsx change, this same legacy-bundle scenario returned no error-kind signal to the client at all — confirmed by inspecting attachRunDetailReadModelFields prior to the fix.

Test plan

  • bun run build
  • bun run typecheck
  • bun test apps/cli/test/commands/results/ (249 pass)
  • bun test apps/dashboard/src/lib/result-table.test.ts apps/dashboard/src/components/ResultTable.test.tsx apps/dashboard/src/components/transcript-timeline.test.tsx (26 pass)
  • Live dogfood eval + CLI results show/summary/report
  • Dashboard serve API verified against both new slim and synthetic legacy bundles

🤖 Generated with Claude Code

index.jsonl rows dropped the inline target_execution envelope in favor
of a compact target_error_kind scalar (av-cpl5.3). Dashboard's
ResultTable read the old nested shape only, so the target-error-kind
badge went dark for new runs. Wire target_error_kind through
serve.ts's row read-model (with a fallback to the legacy nested
target_execution.error_kind/errorKind for older fat bundles) and fix
the Dashboard-side helper to prefer the new field.

av-cpl5.1
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying agentv with  Cloudflare Pages  Cloudflare Pages

Latest commit: 031c9fe
Status: ✅  Deploy successful!
Preview URL: https://99579b00.agentv.pages.dev
Branch Preview URL: https://cpl5-dashboard.agentv.pages.dev

View logs

@christso
christso merged commit 45f7ed1 into main Jul 6, 2026
8 checks passed
@christso
christso deleted the cpl5-dashboard branch July 6, 2026 09:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant