CL experiment v1 — raw data
Four cycles of the continual-learning loop (Aider + qwen2.5-coder:7b on a frozen 24-task FastAPI suite).
Contents: canonical episodes.jsonl, per-cycle raw + gate episodes, aider chat captures, verify logs, distilled artifacts (candidates + promoted), promotions.jsonl, reports (scoreboard, cl-report), the frozen suite incl. hidden tests, and version locks.
Scoreboard: see reports/scoreboard.md inside the tarball.
Lab report / essay: https://agenticfrontier.dev/articles/closing-the-loop
| cycle | completion | mean reward | gate verdict |
|---|---|---|---|
| 1 | 14/24 (58%) | +0.56 | promote v1 |
| 2 | 16/24 (67%) | +0.59 | reject v2 — legacy regression 0.25 > 0 |
| 3 | 16/24 (67%) | +0.65 | promote v3 |
| 4 | 15/24 (62%) | +0.62 | promote v4 |
Reward slope +0.026/cycle; plasticity 1.07. Three quarantined baseline replicates all scored exactly 14/24 (noise band width 0 — the cycle-2 lift is outside run-to-run noise), and a clean-slate reproduction made the same four gate decisions, including the rejection. Honest effect statement: +2 to +4 tasks over baseline while artifacts are active — a direction, not a point estimate.
Integrity
This tarball was assembled at publish time; its SHA256 is
12fbee8402c5407ed2ea0feb955ff570c2f330c7e453a5c50c418543dcbd422a.
The canonical artifacts inside verify byte-for-byte against the pre-publish audit manifest committed in-repo at experiments/v1/overnight/audit/checksums.txt:
episodes.jsonl—51bb562d9cd5d766136c51fc604f2e35dd021fa147bd876799cd08587d6ae32bpromotions.jsonl—b9344de7f78e115a9a3e3ab5996aa58f20c906fcf52c7bd51eaea957acbfb141reports/scoreboard.md—afc5cb3d84173658901d721d778ed7ca6a8ec2400371365adb490c6996e360fd
The overnight self-audit (28 checks, 0 failures), baseline-variance study, and full reproduction comparison live in experiments/v1/overnight/{AUDIT,VARIANCE,COMPARE,REPORT}.md. Suite frozen at tag suite-freeze-v1; run-1 data at tag run1-results.
Reproduce
experiments/v1/ is the runbook: services up → validate → calibrate → freeze → run cycles → publish. ~10 min/cycle on a laptop; an afternoon reproduces the table. Limitations are owned in the essay ("What this does not show"): within-suite improvement only, memory-not-weights, ±2-task post-artifact nondeterminism, n=6 valid split, one model/one harness.