Skip to content

v1.4.0

Choose a tag to compare

@github-actions github-actions released this 16 Oct 23:36

Quality-evaluation release: blind voting that actually works, automatic suite
checks, an LLM judge, OS-keychain secrets, and Elo rankings.

✨ Added

  • Blind voting mode: toggle in the footer (before running), auto-shuffle of columns on every blind run, identities masked everywhere, auto-reveal on vote
  • Test Suite automatic checks (contains/regex/exact/valid-JSON) with per-cell verdicts, detail badges, and pass-rate summary
  • LLM-as-judge panel: blind scoring against a fixed rubric (accuracy/completeness/clarity), deterministic settings, defensive JSON parsing, advisory pick badge that never auto-votes
  • OS-keychain storage for API keys (keyring crate + 3 Tauri commands): auto-migration, in-memory cache, silent fallback; secrets centralized in services/secrets.ts
  • Leaderboard Elo ratings from pairwise vote outcomes (a 4-model win moves more points than head-to-head)
  • Thinking traces persist (reasoning DB column, schema v4) and appear in suite detail, Markdown/PDF exports, and history reloads

🔧 Changed

  • ESLint (type-aware + react-hooks) + Prettier configs, zero warnings/errors enforced in CI
  • Release workflow: code-signing scaffolding (commented, awaiting certificates)
  • History cards use explicit source/provider_id columns (local openai/* models no longer misread as cloud)
  • TypeScript major updates ignored by Dependabot (incompatible with typescript-eslint 8)
  • jsPDF/html2canvas load lazily on export (main chunk 1.49MB → 1.06MB)
  • Release notes drafted from CHANGELOG via scripts/extract-notes.cjs

🐛 Fixed

  • Keychain service namespace fixed in Rust; secret-storage badge (OS keychain vs local) in Cloud header