0.5.0 — the plugin surface

0.5.0 — the plugin surface

AgentDescent as a plugin for Claude Code, Codex, OpenCode and DeepSeek
Harness: a shared skill, an MCP server over doctor / plan / start / status /
show / apply / cancel / resume, a run store so a run started from an agent can
be inspected from a shell, and plugin as an evolvable kind.

Merging is the default now wherever a model is reachable to merge with, so
several workers merge their edits rather than one winning and the rest being
dropped.

v0.4.6 — two benchmarks this repository did not build

Two benchmarks this repository did not build, and a prior that steers the search.

ERA on LLM-SRBench (#159)

examples/era/era_llm_srbench.py — the same flat-PUCT tree, aggregator, sandbox
and governance layer, scored on somebody else's published problems, metrics and
leaderboard. On LSR-Transform (111 Feynman equations rearranged so the closed
form is not one a model memorised), under the benchmark's own per-problem
protocol:

LSR-Transform, all 111	SA	Acc(0.1)	median NMSE	budget
LaSR (paper, best backbone)	6.31%	50.45%	0.0011	millions of GP mutations
LLM-SR (paper, best backbone)	31.53%	39.64%	0.0091	250 prompts
here, deepseek-v4-flash	41.4%	56.8%	2.15e-08	16.5 calls

Five protocol settings are stricter than the benchmark's own, so those are floors.

ERA on AlgoTune (#160)

examples/era/era_algotune.py — 147 of AlgoTune's 154 tasks, one tree each,
scored in speedup rather than accuracy, with the task's own reference as the
root node and its own is_solution as the oracle.

A model prior in PUCT's P(s,a) (#160)

--prior-exponent (default 0.0, upstream's uniform 1/N to the
floating-point bit) asks the mutation reply for a PROMISE: <n> rating of the
approach after tuning, and uses p^k / Σp^k as the prior. Read out of a reply
the port already pays for, so it costs no extra call.

Over 250 rated nodes, 60 of the 137 rated ≥8 reached 2x against 3 of the 93
rated ≤6 — Fisher one-sided p=2.2e-13. Against validity it is worth
nothing (Spearman 0.046), which is the right division of labour when the sandbox
already measures correctness.

On the eight tasks AlphaEvolve, MetaEvolve and OpenEvolve all publish, one run
each: harmonic mean 2.195x against AlphaEvolve's 1.392x and MetaEvolve's
2.045x, ahead on five of eight against each; 1.440x with the uniform prior.
Three caveats travel with that number and are in
bench/results/era-algotune-model-prior.md
— chiefly that lu_factorization's 4.464x skips the reference's .tolist()
rather than factorising faster (discounting it gives 1.782x), and that it is one
run per task on a benchmark whose spread on polynomial_real alone covers
0.983x to 962x.

Fixed

• The AlgoTune sandbox capped the reference's threads and not the
  candidate's. None of the BLAS variables reach numba, which reads
  NUMBA_NUM_THREADS, so an @njit(parallel=True) candidate was timed on every
  core against a one-core reference. Neither side is capped now — writing a
  parallel implementation is an optimisation — and --cpu-seconds scales with
  the core count.
• A rejected answer could be told its numbers were correct. The note
  attached to an is_solution rejection flattened both sides before comparing,
  so right values in the wrong container looked identical to it.

Full notes: CHANGELOG

v0.4.2 — where those fifteen rows actually came from

A follow-up to 0.4.1, which fixed a dropped flag but left the wrong attribution
in place. Full detail in
CHANGELOG.md.

Fixed

• A run reported what it reached and nothing about how it was set up, so fifteen recorded rows attributed themselves to the wrong lag budget. Every bench/results/*.json written from a MethodPolicy port pairs a cells list with a hand-typed config block; the cells were transcribed from the line standard_main prints, which states results only. Fifteen blocks recorded async_ratio: 2 for runs that took run_port's default of 1, and the attribution named bench.candidate_methods as the source.

  It was not. That harness has no --staleness and never passes staleness= to run_port, so every run through it is guarded — and all fifteen blocks record full. The transcribed line is what produced them, and that is checkable rather than argued: it formats qualities at .3f, seconds at .1f and calls as an int, and across all 45 cells not one value carries more precision, while the one file bench.candidate_methods did write has 198 of 198 wall/engine values at full float precision. Those files now record 1, with a note, and the eleven Run it commands pass --async-ratio 1.

• run_port records staleness in framework — the one setting a run could not state about itself, and the one that made those blocks impossible to attribute either way.

Added

• examples._method_runner.run_config and the config: line. A live run prints its resolved configuration as one JSON object, in the key names those results blocks already use, so a block is copied out of a run instead of remembered about it.
• tests/test_results_provenance.py keeps the forensic invariant executable: a future results file whose cells carry more precision than that print line came from somewhere else, and its config block cannot be read as describing a command-line run.

v0.4.1 — flags that were declared and never read

Everything that had accumulated since 0.4.0. Full detail in
CHANGELOG.md.

Fixed

• Four shared flags reached the MethodPolicy runner and were dropped. --async-ratio, --eval-concurrency and --eval-cache were declared by the shared parser and never passed to run_port, so on all eleven declarative ports a run that set every one of them was byte-identical to one that set none; --val-cap parsed and moved nothing, because those ports freeze their splits in build(). All three are threaded and recorded now, --val-cap is withdrawn, and tests/test_method_runner_flags.py enumerates the parser and fails on any flag with nowhere recorded that reads it. (#143)
• A work-budget stop on the async path abandoned the evidence it had paid for. When max_rollouts tripped, up to n_workers - 1 rollouts were still in flight with their model calls already billed, and the merger drained the intake once and returned. Measured on an 8-worker GEPA run: 8 cards produced, 7 abandoned.
• evolve(policies=..., aggregator_factory=...) silently ignored the merge-side policies. It now warns, naming the dropped ones.

Added

• Usage.failure_seconds — model time spent inside calls that ultimately failed, kept apart from seconds so wall - failure_seconds is the wall-clock net of endpoint weather.
• bench.matrix_run — the parallelisation-matrix runner: one row per port, serial vs N-wide arms, --budget-rollouts pinned on both, results written after every cell.
• GEPA --reflective-merge merges the round's diffs into one pool candidate. At equal budget (16 rollouts, seed 0, net of failures): serial 1424s/97 calls, sync N=8 1022s/70 calls (−28%), test EM within noise across arms.
• Shared port flags: --eval-concurrency, --val-cap, --reflective-merge, GEPA's --seed-instruction.

Changed

• The fusion tournament is off by default; the union goes straight to the acceptance gate. Measured on the multi-key fixture: 88 → 48 task evaluations, same final_reward=1.000. evolve(fusion_tournament=True) restores ranking.
• cheap_eval_tasks=None now means 8, or the whole held-out set when smaller — ThreeLayerVerifier.rule_subset's own default, which evolve() had been overriding. Pass len(held_out) for the old behaviour.
• OpenEvolve's --serial now means the shared thing (one worker).

v0.4.0 — one engine

One engine. 0.3.0 made the numbers readable; this release removes the second
implementation they were being measured against.

AgentDescent and AsyncAgentDescent each had their own round barrier, worker
dispatch, snapshot staggering, merger thread, published head and backpressure.
docs/architecture.md called that "a known wart rather than a design intent", and
it kept costing: two measured fixes that had to be hand-ported between the loops, two
early-stop epsilons nobody chose, and three mechanisms the general engine
re-derived — and got wrong — because the reference stack already had them.

Both are adapters over evolve() now. The public surface is unchanged, the
sequential barrier is preserved so a seeded run stays reproducible, and the
reference table still reads first 0.604, final 1.000, against a fork baseline of
0.379.

Highlights

• One engine, two runtimes. The reference domain is expressed in the vocabulary
  evolve() / async_evolve() speak, and Worker is gone. The three things the
  translation does not preserve exactly are listed in domains/router.py rather than
  left to be discovered.
• An execution-and-resource plane, with a page of its own. Sandbox leases with an
  owner, a ceiling and a way to be reclaimed; a real container boundary verified
  against both podman and docker; a ledger and an evaluation cache that survive more
  than one writing process; ClusterParallel and the feedback channel a scheduler
  needs.
• The executor seam is actually wired. A supplied Policies(executor=...) used to
  run nothing while the run reported that it had gone fine — on both paths.
• Replaceable policies. Each merge, staleness, retirement and early-stop decision
  is a contract you can swap without changing any of the defaults.
• Evolving a directory — a skill folder, an agent folder, or its code — with
  FileTree(frozen=[...]) enforced twice.
• Ports share one tested CLI contract, each owns a directory, and the results
  table now marks which rows survive a change of model.
• Docs rebuilt as a reference, with a generated API page and one table of every
  primitive that is provided, tested, and not in any engine path.

Breaking

• Removed: agentdescent.worker.Worker, AsyncConfig.aggregator_interval,
  AsyncConfig.worker_pause, EvidenceCard.version_annotations, the
  domains.router.Task alias.
• full_eval is no longer part of the Evolvable protocol (implementations keep
  theirs).
• Evolvable.cheap_eval → evidence_eval; cheap_eval remains an alias, and the
  moved strategy import paths still work.

Each entry in the changelog says what replaced it.

Not claimed

Two of the five re-measured port results are knob-dependent — --hard and
--top-k are calibrated against the published model — and docs/results.md marks
those rows ⚠︎ rather than presenting them as reproductions.

Full detail: CHANGELOG

v0.3.0 — a measurement pass

A measurement pass over 0.2.0's honesty pass. The thread running through it: on a
reasoning model, too small a token budget returns empty visible content, which
does not raise — it scores as a wrong answer. A starved run therefore reports a low
accuracy indistinguishable from a model that cannot do the task.

ADAS's meta-agent hit this on every call (0 of 4 replies at max_tokens=4096;
4 of 4 at 16384), so its search proposed nothing and the docs recorded
"no lift demonstrated" for a cause that was never the algorithm.

Highlights

• Empty completions are no longer silent. --max-tokens / --timeout, blank
  replies counted and warned about, and a pre-flight check that sends a reasoning
  prompt and aborts if it comes back empty.
• ADAS can now measure something. The hard subset grew from 47 items to 222
  (full MGSM, 11 languages, direct accuracy 0.919), split 34 / 110 / 78 and
  stratified by language; the run reports searched-vs-best-seed rather than a lone
  number that says nothing on a --hard subset.
• The version could not drift, but was pinned to the wrong number. 0.2.0
  single-sourced the version onto agentdescent.__version__, which said 0.7.0.
  This release would have published 0.7.0 to PyPI and burned 0.3.0–0.7.0
  permanently. Fixed.
• Port fidelity against the upstream repos, a stable public API surface, one async
  pipeline instead of two divergent ones, and experiments that measure something.

Not claimed

ADAS's searched-vs-seed lift is still unmeasured — docs/algo-adas.md shows the
comparison with that row explicitly empty. Everything else in docs/results.md
is a completed before → after.

Full detail: CHANGELOG

v0.2.0 — correctness and honesty pass

A correctness pass over 0.1.0: 42 commits, 64 files, test suite 148 → 272.

Almost nothing fixed here was a crash. The recurring pattern was the silent
wrong — code that ran, returned a plausible number, and misled you.

Fixed (selected)

• Flags accepted and ignored. self_verify=False did nothing on the sync path
  (quietly doubling the LLM cost of every proposal); max_seconds bounded nothing.
• oracle_budget capped nothing — it counted calls while the evaluation ran regardless.
• A probability reported as a reward. async_evolve put the Beta-posterior
  P(Δ>0) into held_out_reward, so history was fiction and target_reward could fire on it.
• Seeded runs were not reproducible across processes — hash(str) is randomised
  per process and was used to seed workers and assign partitions.
• A wrong-scale scorer learned nothing, silently. A 0–100 reward made every task
  look solved: propose() never ran, yet final_reward came back as 85.0.
• TensorParallel was not tensor parallelism — section ownership was never
  enforced, so it was DP wearing TP's name. Now out-of-section edits are rejected.
• The reference runtime had no error handling at all — on a dead backend it
  burned its whole budget and returned accuracy=0.000 as a normal result.
• Ledger CAS could be bypassed by a writer declaring no base version.
• Silent document truncation in the document adapter, measured on real 390 KB files.

Documented rather than pretended

ResumeQueue is never popped, commit_atomic is never called, the tail canary set
and staged L1 rollout do not exist, and async_evolve ignores parallel=. Where a
claim could be made true it was; where it could not, the docs now say so.

Added

• One Completion contract for every backend — API models, CLI agents
  (claude_code(), codex(), cli_agent(...)) and OpenHands, with WorkspaceAgent
  for staging files and document_agent as an explicit domain adapter.
• Usage — real token and cost accounting from the provider responses.
• on_round progress callbacks, round_timeout, task samplers,
  EvolutionResult.save/load, EvolutionResult.error, and a documented resume story.

Performance

Ledger reads no longer fork a git checkout per call: 18.7 ms → 0.02 ms, and
run_demo end-to-end 4.74 s → 2.21 s with identical results.

Also

The README Quickstart and the usage guide's examples now actually run — tests
execute them so they cannot rot again. pip install users are told the examples
need a checkout.

v0.1.0

Full Changelog: https://github.com/Birfy/agentdescent/commits/v0.1.0