You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The report JSON config contains llm_model and llm_provider (e.g., claude-sonnet-4-6 / vertex-anthropic), but neither the HTML report nor the CLI summary displays the judge model name.
The judge cost line shows Judge: 20 calls, 41,861 in / 11,558 out tokens — but not which model produced those scores. Users need to know which judge model was used, especially when comparing reports or evaluating calibration.
Current State
config.llm_model and config.llm_provider are in the report JSON
The HTML report shows Judge: with token counts but no model name
The diff module compares judge configs and warns on mismatch, but the individual report view doesn't show the judge model at all
Proposed
Show judge model alongside the cost line:
CLI:
Judge: claude-sonnet-4-6 (vertex-anthropic) · 20 calls, 41,861 in / 11,558 out tokens
HTML: Add judge model to the header meta section, similar to the Agent model field from #179.
Files
src/raki/report/cli_summary.py — add model name to judge cost line
src/raki/report/html_report.py + templates/report.html.j2 — add judge model to header
Context
Found during v0.9.1 release verification — ran raki run --judge and couldn't tell which model scored the retrieval metrics without inspecting the JSON.
Problem
The report JSON config contains
llm_modelandllm_provider(e.g.,claude-sonnet-4-6/vertex-anthropic), but neither the HTML report nor the CLI summary displays the judge model name.The judge cost line shows
Judge: 20 calls, 41,861 in / 11,558 out tokens— but not which model produced those scores. Users need to know which judge model was used, especially when comparing reports or evaluating calibration.Current State
config.llm_modelandconfig.llm_providerare in the report JSONAgent:model line (feat: distinguish agent model from judge model in reports #179) but no judge model lineJudge:with token counts but no model nameProposed
Show judge model alongside the cost line:
CLI:
HTML: Add judge model to the header meta section, similar to the
Agent modelfield from #179.Files
src/raki/report/cli_summary.py— add model name to judge cost linesrc/raki/report/html_report.py+templates/report.html.j2— add judge model to headerContext
Found during v0.9.1 release verification — ran
raki run --judgeand couldn't tell which model scored the retrieval metrics without inspecting the JSON.