Skip to content

v0.4.4 β€” Catch-up release: Darwin evolution, the Flywheel, security hardening, and 115 PRs since v0.1.3

Latest

Choose a tag to compare

@ruvnet ruvnet released this 10 Aug 19:12
· 72 commits to main since this release
30bf8cb

v0.4.4 β€” Catch-up release: everything since v0.1.3

The GitHub Releases page fell behind actual shipping β€” metaharness has been incrementally published to npm all the way from 0.1.3 to 0.4.4 since June 14, but this is the first Release entry to document any of it. This release covers 115 merged PRs across roughly two months of work. Organized by theme, not chronology, with every claim traceable to a real PR number.

Darwin Mode β€” self-improving harness evolution

The core evolutionary engine matured substantially:

  • Darwin Shield (#41): a defensive zero-day harness, Darwin Mode applied to security response itself (ADR-155…167).
  • Self-learning WASM-memory cost-optimal escalator (#43): tiered compute escalation (E2/E3/E4 cores) with peer-reviewed mitigations (ADR-169).
  • Non-stationary CFR/CFR+ poker solver (#49): Darwin evolving against a genuinely adversarial, shifting-strategy opponent via Chebyshev schedules β€” a harder generalization test than static benchmarks.
  • GEPA Distilled Executor Genome (#64) + Agenticow branch-memory for GEPA lineage (#65): offline evolution of the cheap-executor policy, with full candidate lineage tracked like version-control branches.
  • @metaharness/darwin 0.8.0 (#74): the GEPA engine shipped as a pluggable evaluator (darwin/gepa), with the cand-6 genome landing as the first real promoted candidate (#70).
  • metaharness learn (#72): ADR-235 learning command, $0 dry-run by default, cand-6 seed packaged in.
  • Provably-clean SOTA (#76): an Ed25519-signed integrity gate β€” results are cryptographically attestable, not just self-reported.
  • Solver-trajectory forward-contract (#79): flips several previously-advisory checks (no_gold_in_loop, localization_no_gold, best_of_n_selector_conformant) from skip to hard-enforced.
  • Darwinβ†’Claude-P+Fable hard-tail actuator (#60): a handoff mechanism so Darwin's evolved policies can actually drive a real agentic tail-end, not just score in isolation.
  • Advisor track killed (#82): ADR-226 was marked Accepted/NULL after a measured replay (G3) showed it didn't earn its complexity β€” a documented negative result, not silently dropped.

The Flywheel β€” self-optimizing promotion pipeline

The biggest single cluster of work (#84–#116, ~30 PRs over one week):

  • @metaharness/flywheel launches (#84) with a run|replay|graph CLI, and rapidly gains checkpointing (#102), crash-resumable runs (#103), a mutation-effectiveness analyze command (#104, #107), and dual CJS+ESM builds (#112) for broader consumer compatibility.
  • SWE-bench becomes a first-class flywheel domain (D1 series, #86–#114): a code-repair evaluator adapter, a frozen 40-task holdout + 15-task anchor, a real CLI solver wrapper, the official Docker grader with budget caps, fetch-resilience for flaky runs, a structural capability lever so Darwin evolves actual solver behavior rather than just prompt text, and a $0 local-endpoint path that removes the budget gate entirely for local iteration.
  • Multi-vertical scale-up: HLE (#89), extract (#98), math (#95), SQL (#97), and tool-calling (#96) each get their own flywheel domain adapter β€” the same evolutionary machinery now generalizes across five distinct task families.
  • GSM8K goes live (#115–116): a fast, $0, real-compounding domain testbed, with an explicitly documented "honest null" result on a second real domain β€” the project reports what didn't work as carefully as what did.
  • Rigor discipline: verifyReplayBundle re-executes the gate on sealed scores rather than trusting a cached verdict (#99); honest-null runs are required to replay-PASS under a re-executing-verifier standard (#93, #100) β€” the same "don't trust your own cached success" principle later became this session's ADR-381 work on the ruflo side.
  • Anytime-valid sequential evidence for promotion gates (#159) and receipt-chain export/import/merge for cross-system audit (#85) β€” statistical rigor and portability for the promotion record.

Security hardening sprint (#123–#128)

A tightly-scoped, clearly-labeled batch closing several real findings in one pass:

  • Value-aware secret redaction across every output surface (#123).
  • mcp-scan findings pinned at HIGH severity so a future downgrade can't silently disarm the gate (#125).
  • Publish fails closed if a witness signature was present but never actually verified (#124) β€” the fail-open version of this bug is the dangerous one.
  • {{var}} template injection-safety charset pinned by test, so a future loosening can't slip through unnoticed (#126).
  • Witness shape-gate boundaries pinned by test against a relaxed bound silently accepting malformed witnesses (#127).
  • Every CI matrix job gets a timeout-minutes so one hung job can't block merges for six hours (#128).

Workspace-Lens β€” interpretability primitive

  • @metaharness/workspace-lens (#129): a Jacobian-Lens interpretability primitive (ADR-238).
  • @metaharness/workspace-probe (#131): the eval + Darwin-Mode bridge that turns lens output into mutation-evidence signals (ADR-239).
  • fromUrl/fromRegistry (#132) and a CLI with readout/probe/grade-mutation + runnable demo (#136) β€” go from "load a fitted lens over HTTP" to "grade a specific mutation" in two releases.

Cost & economics

  • Cost-aware output-mode decoder policy (#81): optimizes cost-per-accepted-task, not just raw token cost (ADR-232).
  • BenchPress (#63): low-rank score prediction for model onboarding and tier placement β€” estimate where a new model sits before running the full benchmark suite.
  • RLI economic-workflow oracle (#67): internal target generation as the primary signal, tier-placement calibration as a secondary one.

Version control for agents

  • @metaharness/jujutsu (#56): version-control-for-agents plus an agenticow dual-state bridge (ADR-202) β€” treating an agent's working state with the same rigor as a jj/git working copy.

AGNTCY / Outshift integration (#155–158)

  • ADR-240: identity, OASF export, observability, and a CASA compiler.
  • Real OASF taxonomy mapping and real TLS/x509/JWT auth wiring (#156) β€” not a mock integration.
  • A schema_version mismatch was correctly diagnosed as the real root cause of an earlier failure, not a validator bug (#157) β€” worth calling out because misdiagnosing that class of bug is a common trap.
  • A full end-to-end showcase example (#158).

Meta-Proxy

  • Signed routing + worktree policy (#144), with version pins bumped and a dedicated pin-drift watcher added (#145, #151) β€” the same "stale pin silently drifts" lesson that came up again in this session's ruflo work, addressed here first.
  • Opt-in start-at-login for the sidecar (#164).
  • metaharness itself republished at 0.4.2 (#152) and 0.4.4 (#163) specifically to deliver newer Meta-Proxy pins to npm.

New engines and hosts

  • Kimi-K3 harness (#170): a C-parity Rust/WASM K3 engine, plus AgentRadio and NOOA clones, with Darwin/flywheel optimizations β€” described as "all operational, replay-verified."
  • Prime Agent integration (#169): the 10th supported host, with kernel session + autonomous primitives in both Rust and TS/WASM, and a new RefineMutator (ADR-241/242).

Today's work: recursive turn-level credit assignment

  • @metaharness/turn-credit (#176, ADR-248): converts a single terminal pass/fail verdict into per-turn credit via a recursive belief-update, with a mathematically-enforced bound that reshaping can adjust emphasis but can never reverse the verifier's actual decision.
  • Four provable enhancements landed alongside it, each independently adversarially reviewed: Darwin scorer signal seams (ADR-249, additive/opt-in β€” byte-identical output when unused), a 98-file ADR canonical-section sweep, and the @metaharness/router calibration-audit module (Brier score, reliability bins, expected calibration error).
  • #177 synced darwin/router's package.json versions to what was actually published (0.9.0 / 0.4.0) after the initial merge.
  • Full plain-language write-up, including three concurrency/epoch-boundary bugs a downstream review caught and fixed before the corresponding ruflo integration shipped: https://gist.github.com/ruvnet/0202e6a060b04b3a4f5d9bd18345169d

Also in this window

Ongoing scaffolding/tooling work: a browser-based agent-harness generator (#1), a real WASM kernel backend with diagnostics (#27), 18 third-party SDK showcase examples (#19), bespoke per-SDK agent bundles (#29), and a steady stream of publish/CI fixes (napi packaging, Windows tar bugs, ERESOLVE dependency alignment, rustfmt/clippy/cargo-deny gates, idempotent publish workflow) that don't individually warrant a headline but collectively kept 115 PRs landing cleanly over two months.


Packages touched in this window (current published versions): metaharness 0.4.4 Β· @metaharness/darwin 0.9.0 Β· @metaharness/flywheel 0.1.10 Β· @metaharness/router 0.4.0 Β· @metaharness/radio 0.1.0 Β· @metaharness/turn-credit 0.1.0 Β· @metaharness/workspace-lens 0.1.1 Β· @metaharness/jujutsu, @metaharness/kernel, @metaharness/hosts, and the evals-* / create-agent-harness families, each independently versioned.

πŸ€– Generated with RuFlo