Skip to content

v0.4.1 — flags that were declared and never read

Choose a tag to compare

@Birfy Birfy released this 10 Aug 13:06
· 1043 commits to main since this release
7fede50

Everything that had accumulated since 0.4.0. Full detail in
CHANGELOG.md.

Fixed

  • Four shared flags reached the MethodPolicy runner and were dropped. --async-ratio, --eval-concurrency and --eval-cache were declared by the shared parser and never passed to run_port, so on all eleven declarative ports a run that set every one of them was byte-identical to one that set none; --val-cap parsed and moved nothing, because those ports freeze their splits in build(). All three are threaded and recorded now, --val-cap is withdrawn, and tests/test_method_runner_flags.py enumerates the parser and fails on any flag with nowhere recorded that reads it. (#143)
  • A work-budget stop on the async path abandoned the evidence it had paid for. When max_rollouts tripped, up to n_workers - 1 rollouts were still in flight with their model calls already billed, and the merger drained the intake once and returned. Measured on an 8-worker GEPA run: 8 cards produced, 7 abandoned.
  • evolve(policies=..., aggregator_factory=...) silently ignored the merge-side policies. It now warns, naming the dropped ones.

Added

  • Usage.failure_seconds — model time spent inside calls that ultimately failed, kept apart from seconds so wall - failure_seconds is the wall-clock net of endpoint weather.
  • bench.matrix_run — the parallelisation-matrix runner: one row per port, serial vs N-wide arms, --budget-rollouts pinned on both, results written after every cell.
  • GEPA --reflective-merge merges the round's diffs into one pool candidate. At equal budget (16 rollouts, seed 0, net of failures): serial 1424s/97 calls, sync N=8 1022s/70 calls (−28%), test EM within noise across arms.
  • Shared port flags: --eval-concurrency, --val-cap, --reflective-merge, GEPA's --seed-instruction.

Changed

  • The fusion tournament is off by default; the union goes straight to the acceptance gate. Measured on the multi-key fixture: 88 → 48 task evaluations, same final_reward=1.000. evolve(fusion_tournament=True) restores ranking.
  • cheap_eval_tasks=None now means 8, or the whole held-out set when smaller — ThreeLayerVerifier.rule_subset's own default, which evolve() had been overriding. Pass len(held_out) for the old behaviour.
  • OpenEvolve's --serial now means the shared thing (one worker).