amberfork v1.0.0

the credibility release

amberfork aligns two AI-agent run traces — one that worked, one that didn't — with a real
sequence-alignment algorithm, localizes the exact step they forked at, and (on a run you captured
yourself) can work toward confirming that step caused the failure by replaying it with the fork
patched. Local, deterministic, offline. No account, no API key required for the core path.

Every milestone before this one built the tool. This one tested it — specifically, tested the
claims the tool makes about itself, against real data, published whichever way the numbers fell.
If you're deciding whether to trust this project, docs/writeup.md is the
one page to read; this is the shorter version.

What's new since v0.9.1

The missing baseline finally exists, and it wasn't kind. The project's own headline number
(91% fork localization within 3 steps, on controlled/injected forks) had never been checked against
the obvious question: does a frontier LLM just asked to find the bug do better? It does — three of
five judge arms beat the engine on real natural-failure pairs, with intervals that exclude zero.
Published as measured, not softened. The mitigating half of that same result: no arm, including the
judges, clears random guessing by much either — which points at the corpus more than any one method.

A fourth attempt at natural-fork evidence, built from scratch this time. Every earlier attempt
at finding a natural (non-injected) fork ran into a defect in someone else's dataset. This one
removes that excuse: a real local model drives a real tool-using agent through this project's own
unmodified recording path, with a known, engineered cause. Result: a tie with the simplest possible
baseline — and a real, useful diagnosis of why, not a shrug.

--verify's other half of the pitch, measured for the first time. Confirming a fork's cause by
counterfactual replay had a single hand-written test and no rate. Now it has one: 87% confirmation
across 15 real, independently-forked runs, zero indeterminate verdicts. The two cases that didn't
confirm are the most interesting data in the table — read the writeup for why.

Two documentation-drift issues closed with the reasoning kept, not just the fix — including a
deliberate decision not to build a metric that looked easy but would have quietly compared amberfork
against a number it wasn't actually measuring.

--judge local now defaults to a model that can actually narrate a fork (the old default was
too small to produce useful output). Five release targets, a real CI gate recipe, repo metadata.

What this release does not claim

• The 91% localization number is on controlled, injected forks — real logs, a real spliced
  divergence, honestly labeled as constructed. It is not a claim about natural failures.
• On the one natural corpus with an LLM-judge comparison, amberfork loses to the judges on the
  metric that matters. Read docs/writeup.md for the full, unflattering table.
• --verify's 87% confirmation rate has no independently-known ground truth behind it — it's a
  measure of how often the mechanism reaches a decisive verdict on real sampling-variance forks, not
  a precision score against a known cause. Why that number doesn't exist yet is in the writeup too.
• On two runs that share no common vocabulary, amberfork today reports a fork with high confidence
  while being wrong. The fix is designed, not shipped — the single highest-priority item on the
  open roadmap.

Try it

cargo install amberfork
amberfork serve --demo

Point it at your own traces from there: the trace format,
a worked example. Every number in this release reproduces
offline, from a committed file, with no API key — commands for each are in
docs/writeup.md and README.md.

Full technical detail, every dead end included: docs/notebook.md, now 80
dated entries. Full change list: CHANGELOG.md.

amberfork v0.9.1

Full Changelog: v0.9.0...v0.9.1

amberfork v0.9.0

Full Changelog: v0.8.0...v0.9.0

amberfork v0.8.0

Full Changelog: v0.7.0...v0.8.0

amberfork v0.7.0

Full Changelog: v0.6.0...v0.7.0

amberfork v0.6.0

Full Changelog: v0.5.0...v0.6.0

amberfork v0.5.0

Full Changelog: v0.4.0...v0.5.0

amberfork v0.4.0

Full Changelog: v0.2.0...v0.4.0

v0.2.0 — offline benchmark

v0.2 — offline benchmark (milestone issues #6, #7, #11).

• amberfork-bench: the pre-registered chimera protocol (BENCHMARK.md) as a Rust harness —
  four arms (random, pos-lexical, nw-structural/resync, nw-lexical/resync) scored on identical
  fixtures with coverage accounting, Wilson 95% intervals, and a calibration curve; parameter
  freeze via bench/params.toml (every table names the config sha256); committed results
  documents + offline report mode that re-renders them with zero fetch.
• Mode A′ cross-system pairs, end-to-end in one binary: fetch (pinned upstream commits,
  licensing notices, provenance record) → TapeAgents reference adapter → build-pairs →
  scored disclosure. The honest result on the 4 constructible real pairs is a null
  (engine 0.50 ±3 vs random 0.75 — short runs, murky cross-system gold; both pre-registered
  threats) — shipped as a disclosed limit, not a headline (notebook 016).
• CI-visible fork-localization gate: GAIA-sanitized dev fixtures for seeds 42/43/44 committed
  under bench/fixtures/chimera_noise_seed*_dev/ with a full regeneration recipe;
  chimera_parity pins each seed's own baseline on every amberfork-align change (issue #11,
  notebooks 013–014).
• Honesty correction: the seed-42 0.75 exact headline was a favorable draw — README and the
  gate now lead with the cross-seed ±3 window and the 0.56 aggregate exact (notebook 014).
• First sealed test-split reveal (protocol rule 2, at this tag): dev-tuned params
  generalized — engine ±3 0.91 [0.78, 0.97] vs best baseline 0.49 [0.33, 0.64]
  (non-overlapping), exact 0.49 [0.33, 0.64] vs 0.00, on n=35 unseen pairs across three seeds;
  calibration monotone on unseen data; results committed with report snapshot tests
  (notebook 017).
• amberfork-ingest: warn when a trace declares a non-native schema version.
• Workspace version now single-sourced from [workspace.package] (0.2.0).

v0.1.0 — walking skeleton

v0.1 — walking skeleton (milestone issues #1–#5): amberfork diff <bad> --against <good>
working end-to-end, thin everywhere.

• amberfork-model: canonical Run/Step trajectory model + DiffResult (the frozen,
  schema_version-carrying contract every consumer reads).
• amberfork-ingest: plain-JSON trace loader (docs/trace-format.md) + Who&When converter.
• amberfork-align: lexical/tf-idf cost model, affine-gap Needleman–Wunsch, resync fork rule
  (first non-sync block the alignment never recovers from); self-align invariant as the
  canonical guard.
• amberfork-cli: amberfork diff with amber terminal render + --json; amberfork demo
  runs a bundled pair embedded in the binary; README hero GIF from a committed vhs tape.
• Pre-build spike (2026-07-07, throwaway Python, findings only): alignment + sustained-
  divergence localizes at 70% exact / 90% ±1 under benign noise vs 0% for positional
  first-mismatch on real Who&When content — the core bet validated before the first crate
  (notebook 001; consequences: resync fork rule, embeddings demoted to a hypothesis,
  benchmark reframed to controlled-injection + Mode A′).
• Pre-registered benchmark protocol + dataset licensing resolution (BENCHMARK.md); canonical
  plain-JSON trace format; project scaffolding (README, MIT license, CI running the exact
  local verify gate, engineering notebook).