Skip to content

feat: show judge model name in HTML and CLI report headers #207

@decko

Description

@decko

Problem

The report JSON config contains llm_model and llm_provider (e.g., claude-sonnet-4-6 / vertex-anthropic), but neither the HTML report nor the CLI summary displays the judge model name.

The judge cost line shows Judge: 20 calls, 41,861 in / 11,558 out tokens — but not which model produced those scores. Users need to know which judge model was used, especially when comparing reports or evaluating calibration.

Current State

  • config.llm_model and config.llm_provider are in the report JSON
  • The CLI summary shows the Agent: model line (feat: distinguish agent model from judge model in reports #179) but no judge model line
  • The HTML report shows Judge: with token counts but no model name
  • The diff module compares judge configs and warns on mismatch, but the individual report view doesn't show the judge model at all

Proposed

Show judge model alongside the cost line:

CLI:

Judge: claude-sonnet-4-6 (vertex-anthropic) · 20 calls, 41,861 in / 11,558 out tokens

HTML: Add judge model to the header meta section, similar to the Agent model field from #179.

Files

  • src/raki/report/cli_summary.py — add model name to judge cost line
  • src/raki/report/html_report.py + templates/report.html.j2 — add judge model to header

Context

Found during v0.9.1 release verification — ran raki run --judge and couldn't tell which model scored the retrieval metrics without inspecting the JSON.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions