v0.2.0 — Whitepaper v0.3 + AgentMeasure Lab v0.4 (open experiment engine)
What's in this release
AgentMeasure Lab v0.4 — the open experiment engine for the agent channel. Zero dependencies (Python 3.9+ stdlib only), fully offline, no registration.
python3 lab/am lab selftest # planted uplift recovered + honest null, in seconds
python3 lab/am lab init # workspace + example experiment
python3 lab/am lab preregister am-lab/experiments/example-manifest.json
python3 lab/am lab run am-lab/experiments/example-manifest.prereg.json- Preregistered experiments (hypothesis / primary metric / guardrails / analysis plan hashed before the run; tampering refuses to run) with a scale / power / budget preview
- Honest statistics: Wilson/Newcombe intervals, honest nulls with next-round sizing guidance, "undetermined" with required n — never an underpowered verdict
- Fake-growth rejection at the decision exit: a selection uplift that loses consumption is verdicted
unverified_growth— do not ship; margin is computed with the measured (lower) consumption - Guardrails & dominance: significant wins that breach guardrails are
effective_not_qualified; candidates that make no more money at higher cost are called out as dominated - Bilingual decision-maker one-pager at the top of every report (conclusion / uplift / monthly margin / certainty / recommended action)
- Calibration loop (
am lab calibrate): offline vs production re-test, per-condition transfer effects with intervals — never a single global transfer coefficient;not_comparablegaps are named, never assumed - Connector data plane (
am connector): three-tier per-class authorization, immediate revocation, HMAC-signed aggregate-only exports - Read-only MCP interface (
am mcp serve) so agents and CI query the same evidence engineers see - Real harness adapters:
claude-code(full; integration-tested against scripted transcripts — first live runs are validation runs) andcodex(experimental), with candidate-set injection via a local MCP tool server - Open formats (FMT-001..004) and a Core 0.4.4 mapping doc; local experiment history (
am lab history); 74 tests
Whitepaper v0.3 (EN + zh-CN, structurally in sync): experimentation & calibration as first-class measurement semantics; anti-fake-growth elevated to a core thesis (with the x402 wash-trading evidence); data-rights claim tiers in the observation-surfaces chapter; the value formula bridging measurement to margin; new evidence (Hasan et al. arXiv 2602.14878, BiasBusters ICLR 2026, Arcade ToolBench).
Also: baseline funnel in product/local-analytics.py (with explicit provider-side observability boundaries), README front-door for Lab (EN/zh).
Honest limits, stated up front: the shipped demo runs on a synthetic harness with planted ground truth (validates the engine, not real-agent claims); production ingestion requires a data-rights agreement (G0); adapter live-CLI validation is pending.