0.45.0
Fixed
- Keep the AEP 0.43.0 rendered-plan change summary within the shared
b10x-change/v1limit so the deterministic organization documentation snapshot can ingest it.
Added
-
aep eval runrefuses to spawn against a stale childaep(EVAL-RUN-017). The session metaharness launches runs on a constructedPATH—$HOME/.local/bin:/usr/local/bin:/usr/bin:/bin— so theaepa case's task executes is whatever sits in~/.local/bin, not the one that launched the run. On 2026-09-03 a 0.40.1 copy there, beside the 0.44.0 that launched, made step 8 of the golden path stop atunrecognized subcommand 'doctor'after $10.96 had been spent. Before a live spawn the runner now resolvesaepthe way the child will and compares its--versionwith its own; a mismatch is refused with both paths and both versions, an absence is a printed warning (the child may not need it), and--streamingestion is untouched.task installrefreshes a real~/.local/bin/aepand~/.local/bin/protocolafter the cargo install for the same reason. -
A case whose
subject.skillsnamesess-schema:*needsesson the child'sPATH(EVAL-RUN-018), or its step is drafted by hand and never validated — which is what the same recording showed. -
aep eval run --budget-usdis now also the session's own ceiling. Each spawned Claude run receivesmetaharness run claude --max-budget-usd <what is left of the cap>, which the vendor enforces during the run; the runner's between-runs check stays. Until now the cap was a receipt: the golden-path case stated $10.96 against--budget-usd 5and the only thing the number did was appear in the report. Claude Code only, on--model's reasoning; needs metaharness 0.6.1. -
aep eval run --model <MODEL>, forwarded verbatim to themetaharness run claude --modelinvocation. The flag states and the harness resolves: nothing here normalises an alias or picks a model, and an invocation that pins none produces the argv it produced before the flag existed. It is placed before the arm's treatment, on every arm, because a model is a condition a phase holds fixed across its arms.- The manifest records
model_requestedbesidemodel.modelstays what the attestation reported;model_requestedis what the run asked for, written immediately after it and only where the run asked for something — so every manifest assembled before this keeps its bytes.eval matrixreads the manifest and gains no column: a phase that fixed a model checks it by reading both fields, and a runner that folded them into one would have thrown away the only evidence that the pin was honoured. --modeloncodexorb10xis refused by name (EVAL-RUN-016) rather than accepted and dropped: their adapters take no model flag at metaharness 0.5.0, and a run that silently used the default would enter the matrix as a run that pinned one.
- The manifest records
-
aep eval run --plugin <repo>@<name>@<version-or-commit>, repeatable, forwarded verbatim to themetaharness run claudeinvocation. metaharness 0.5.0 places a pinned third-party marketplace plugin into its scratch config home and attests it; this runner names one and resolves nothing — no marketplace is read here, no pin is rewritten, and nothing is fetched. It is what a bench arm needs to put a third-party plugin beside a checked-out one through one driver.--pluginand--plugin-dircombine, and armpluginis satisfied by either — so a run whose whole treatment is a marketplace plugin no longer has to name a directory it does not have. Apluginarm naming neither is still refused (EVAL-RUN-012).- The manifest keeps them apart.
plugin_digeststays the digest of what--plugin-dircopied,nullwhere there was no directory; a newpluginslist beside it names each declared marketplace plugin with the digest the attestation stated for it. The key is written only where there is one, so every manifest assembled before this keeps its bytes. - An unpinned value is refused before anything is spawned (
EVAL-RUN-013), in metaharness's own words: an unpinned plugin can change between two runs that both claim to have used it, which makes the two arms of a comparison incomparable and neither of them reproducible.--pluginoncodexorb10xis refused by name (EVAL-RUN-014) rather than accepted and ignored, and on armraw(EVAL-RUN-015), which is the arm with no plugin in it. - A declared plugin the attestation does not list is refused (
EVAL-STREAM-013). The runner declares, the instrument attests, and the manifest records what both said; a declaration nothing attested would enter the matrix as a plugin the run never had.
Changed
aep trace checkdecides a negative expectation only over a stream its producer closed.tool.absent— "this never happened" — can be falsified only by an event, so it can be answeredokonly over a record somebody vouched for; a transcript cut off at any point looks exactly like a run that stopped there. The checker now reads the producer's own statement that the transcript is the whole run and answersunknaming the missing marker when there is none.- What closes a stream, per wire. On
metaharness.event/1: a terminalstream.closedevent as the last line, carrying itseventscount and itsreason(completed|budget|killed|error|steer-halt) into the verdict, orsession.started.hermetic.stream_complete: true. On Claude Codestream-json: the run's terminalresultrecord as the transcript's last line. A Codex rollout carries neither and an absence over one staysunk, which is the honest answer for an append log that says nothing about its own end. session.endedis not read as the metaharness wire's closing record. That wire carries one terminal record per session and a driven run is a concatenation of sessions, so a subagent'ssession.endedwould close a stream in the middle of the run that wrote it. This is the reason metaharness grew a marker of its own.- An unread event no longer vetoes a decided absence, and still poisons every count it could change. A closing statement says the record is whole; it does not say what an unread record was, so
tool.called {exactly: 0}over one is stillunk— but atool.absentrow over a closed stream is decided, and its citation names how many unread events the verdict was reached over. Eight paid runs on 2026-09-03 ended "undecided" on exactly those rows while carrying twenty-three unread vendor lines apiece. - A gap is unaffected. A call that is in the record contradicts an absence whatever else is missing, because reading more of a transcript can only add calls.
- The committed metaharness fixtures under
crates/trace-spec/tests/fixtures/,crates/protocol-cli/fixtures/andconformance/eval/gain the closing line, so their transcript digests moved anddry-run.matrix.jsonmoved with them.
- What closes a stream, per wire. On
aep eval run --redactnow redacts the stream as well as the record. The operator's home directory —$HOMEand its realpath — becomes~, and the operator's user name ($USER,$LOGNAME, and the last segment of$HOME) becomes<user>, in every event text and path field. The eight streams recorded on 2026-09-03 under--redactcarried/home/<operator>between 18 and 49 times each and the user name between 20 and 51 times, so none of them could be committed to the publicrecorded/directory its case was written for.- The substitution is over the file's bytes, not over a list of fields: a home path turns up in an event text, a tool argument, a
cwd, a transcript path and a shell command alike. Home directories are replaced longest first, so a path never degrades to/home/<user>; a user name is replaced at word boundaries only, so an operator calledtimdoes not rewriteruntime. Neither placeholder needs escaping inside a JSON string. - The manifest's
transcript_digestis taken over the redacted bytes, so the file the runner wrote is the file the manifest names and--streamre-ingests it to the same manifest, byte for byte. On the spawn path the redaction happens before the stream reaches the disk. --stream --redactwrites the redacted stream into--outas<run>.events.jsonl, which is how an already-recorded paid run becomes a committable one. Without--redactnothing is rewritten and no stream is written: the caller's file is the record. A refused ingest still writes nothing at all.- Repository names, commit ids and branch names are not redacted: they are the run's subject.
- The substitution is over the file's bytes, not over a list of fields: a home path turns up in an event text, a tool argument, a