Skip to content

[codex] convert evals to cli-evals#8

Merged
kreneskyp merged 1 commit into
mainfrom
codex/convert-evals-to-cli-evals
Jun 27, 2026
Merged

[codex] convert evals to cli-evals#8
kreneskyp merged 1 commit into
mainfrom
codex/convert-evals-to-cli-evals

Conversation

@kreneskyp

Copy link
Copy Markdown
Contributor

Summary

  • add cli-agent-evals.config.mjs using the shared @agent-ix/cli-agent-evals suite API
  • route make evals/evals-all/eval/evals-rebuild through ../cli-agent-evals/bin/cli-evals.js
  • replace the old evals/run.mjs harness with a migration stub
  • document direct CLI usage plus Claude/Codex/opencode/Copilot plugin setup

Validation

  • node ../cli-agent-evals/bin/cli-evals.js list --suite ./cli-agent-evals.config.mjs
  • node ../cli-agent-evals/bin/cli-evals.js rebuild --suite ./cli-agent-evals.config.mjs --report evals/reports/latest.json
  • pnpm run lint
  • live canary via cli-evals: 4/5 passed; EV-013 timed out at HITL gate and is recorded in evals/reports/cli-evals-live-canary-2026-06-27-rerun.json

Dependency

@kreneskyp kreneskyp requested a review from a team as a code owner June 27, 2026 18:57
@kreneskyp

Copy link
Copy Markdown
Contributor Author

CI is green. Merge is currently blocked only by required review (reviewDecision: REVIEW_REQUIRED).

@kreneskyp kreneskyp merged commit a71198c into main Jun 27, 2026
6 checks passed
@kreneskyp kreneskyp deleted the codex/convert-evals-to-cli-evals branch June 27, 2026 19:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant