Skip to content

v4.5.1 — Fix 4 Highest-Frequency Bugs

Choose a tag to compare

@raphaelchristi raphaelchristi released this 03 Apr 00:42
· 132 commits to main since this release

What's Fixed

4 bugs that hit 100% of real-world evolution runs (verified across 3 independent repos).

{input_text} placeholder (3/3 repos affected)

run_eval.py used to replace {input} with a temp JSON file path. Agents expecting --query "What is Kotlin?" would instead get --query /tmp/tmp12345.json and "research" a file path.

Fix: New {input_text} placeholder extracts plain text from the inputs dict, shell-escapes it, and passes it directly:

# Before (broken for text-expecting agents):
entry_point: "python run.py {input}"     # passes /tmp/tmp123.json

# After (works):
entry_point: "python run.py {input_text}" # passes 'What is Kotlin?'

Worktree file propagation (3/3 repos affected)

.evolver.json and .env are untracked by git, so they weren't present in worktrees. Every repo had to manually copy them.

Fix: Evolve skill now auto-copies both files into each worktree before evaluation.

Canary preflight (1/3 repos — burned API quota)

160 Gemini API calls wasted on a broken agent that crashed immediately.

Fix: run_eval.py now runs 1 example as a canary before the full evaluation. If the canary fails (no output + error), exits with code 2 immediately. Disable with --no-canary.

Baseline LLM-judge (2/3 repos — inflated scores)

Baseline scored 1.0 from has_output only (any text = perfect score). Evolution thought the agent was already perfect and stopped.

Fix: New step 0.7 in evolve skill checks if baseline has LLM-judge scores. If not, triggers the evaluator agent before the loop starts. Baseline score drops from inflated 1.0 to actual quality.

Install

npx harness-evolver@latest

🤖 Generated with Claude Code