Skip to content

Mino chapter-11

Choose a tag to compare

@github-actions github-actions released this 26 Oct 11:56
· 15 commits to main since this release

Added

  • Chapter 11 evals: internal/eval runs declared YAML cases through the real Agent
    loop and grades them with deterministic checks (final_text_contains,
    final_text_not_contains, final_text_matches, tool_called, tool_not_called,
    tool_order, result_status, file_exists, file_contains, max_requests,
    max_tool_calls). A case marked live: false asserts a scripted program path and
    is skipped by --live.
  • mino eval [PATH] runs every *.yaml case in a directory (default ./evals). Replay
    mode drives the loop with a scripted loopback Responses service: no key, no network,
    a fresh temporary workspace and home per run, the manual permission mode, and no MCP
    servers. --live uses ~/.mino/config.json, runs each case several times (--runs,
    default 3), and reports pass rate, pass@k, pass^k, and token usage. --filter, --out
    (report.json plus one session log per run), and --timeout refine a run; any
    failing case exits with status 1.
  • A repository evals/ suite covering approved and denied Bash, a protected .env,
    file creation, and an unknown tool. scripts/check.sh runs it after the build.
  • The Agent emits a response_completed event carrying the service's token usage after
    each validated response. The terminal gateway ignores it; mino eval sums it.

Changed

  • The chat banner reads Mino - Chapter 11: Evals, and mino help lists eval.

Fixed

  • Book build: override Vite with 6.4.5 and refresh the lockfile (dompurify, source-map-js)
    so npm audit --audit-level=moderate passes again and the tutorial site can be rebuilt
    and published. VitePress stays at 1.6.4; two low-severity KaTeX advisories remain.