Skip to content

v5.0.1 — Fix 6 Findings from Codex Reviews (GPT-5.4)

Choose a tag to compare

View all tags
@raphaelchristi raphaelchristi released this 03 Apr 03:39
· 104 commits to main since this release
v5.0.1
5b44afd

What's Fixed

6 findings from two Codex (GPT-5.4) reviews — the first cross-model review of this codebase.

From Adversarial Review (3 high-severity)

  • Held-out comparison fails hard on zero examples — read_results.py exits with error when --split filtering leaves 0 examples. Missing held_out is now a blocking preflight error.
  • Canary fails on empty output — No more silent pass when agent exits cleanly with no stdout. Empty output = failure regardless of error string.
  • Constraint gate validates JS/TS/shell — node --check for .js, bash -n for .sh, npm test for package.json. Fails closed when tools unavailable.

From Standard Review (3 skill bugs)

  • Subdirectory projects fixed — Eval and constraint paths now resolve $PROJECT_DIR in worktrees.
  • Race condition fixed — wait before evaluator agent ensures all experiments exist.
  • $BEST initialized — State read block before gather step; skips analysis when no baseline.

Install

npx harness-evolver@latest

🤖 Generated with Claude Code + reviewed by Codex (GPT-5.4)

Assets 2
Loading