Measure Every Batch

Measure every batch

Section 7 of the method. Cost per change and what the critic missed, after every batch.

workflow-cost.py reads the per-agent transcripts a Claude Code workflow leaves behind, sums tokens by model, prices them, and prints cost per change against a baseline.

python3 workflow-cost.py <run-id> --prs 4 --baseline-per-pr 13

Point it at a run directory or a bare run id. Prices are list prices in the PRICES table in lib/leanstats.py; edit them to whatever you actually pay. It imports the price table and the transcript reader from lib/leanstats.py, so copy both. asset-provenance.py stands alone.

lean-stats covers what workflow-cost.py does not: agents started with the Agent tool, and the main session.

lean-stats collect --since 2026-10-01 owner/repo
lean-stats report --since 2026-10-01

collect imports transcripts and merged pull requests. report prints the numbers the retro starts from. An agent is counted by role only when it carries a label, lean:<role> <owner/repo>#<pr>, as its Agent description or as the first line of a workflow prompt. The method and adversarial-review skills write that label. Without one the agent is counted as other, and the report's last line says what share of tokens that was.

Both tools count a message's usage once. Claude Code writes one transcript row per content block and repeats the message's usage on each. Until 1.4.0 workflow-cost.py summed every row, which overstated each of the 17 runs it was rechecked on by 65% to 304%. Its price table was also out of date: Fable was priced as Opus, and Opus and Sonnet at the prices of their version 4 models. Counted once, at the list prices of 2026-10-02, the three baseline runs come to 225 dollars over 18 changes, 13 per change, and the first lean run to 17 dollars over 2 changes, 9 per change: 31% under, where the old figures said 61%.

After each batch, compare two things: cost per change, and what the critic caught versus what got past it. When the critic misses something, the fix is a new scripted check, not a more expensive critic. When a critic finds nothing on a tier for three batches, drop it from that tier.

---

Previous: Rework and concurrency | Home | Next: Attack your own method