KPIs and Metrics

KPIs & Metrics

Status: Complete · What a team of one actually measures. Module 5 introduces Context Entropy as the sizing unit; this page is the full measurement system.

Legacy metrics (story points, velocity, burndown, lines of code) measure collective human effort and coordination. With N = 1 and free code generation, they're noise. The SAE measures flow, quality, and spec health — and guards against gaming each one.

This kit is also the community's evidence engine. SAE is stated honestly as a falsifiable hypothesis with instrumentation, not a proven law — its weakest point is the empty evidence base. The way that changes is operators running the loop and publishing their before/after scorecards. The four-metric scorecard below is the common unit: capture it, and add your numbers to Solo Production Case Studies. The framework earns its claims by being measured, not asserted.

---

The metric system

Group the metrics into three questions: Am I fast? Am I correct? Is the system healthy?

Flow — am I fast?

KPI	How to measure	Healthy direction	Watch out for
Idea → Production cycle time	Timestamp from Frame to deploy, per change	↓ trending down	Shrinking by skipping Spec/Guard
Autonomy ratio	% of changes shipped with zero hand‑edited code (spec → generate → ship)	↑ over time	High ratio with rising escape rate = sloppy, not autonomous
Entropy mix	Distribution of task ratings (Low/Med/High)	Mostly Low/Med	All "Low" may mean you're avoiding hard, valuable work

Quality — am I correct?

KPI	How to measure	Healthy direction	Watch out for
Regression escape rate	Bugs reaching production ÷ changes shipped	↓ toward zero	The single most important quality signal — don't trade it for speed
Guardian catch rate	Defects caught in Guard ÷ total defects found	↑ (caught before prod, not after)	A falling rate with steady bugs means Guard is weakening
Rework ratio	Generation cycles needed per shipped feature	↓ (1–2 is healthy)	Consistently high → specs are too vague
Edge‑case coverage	% of spec'd edge cases with a passing test	→ 100%	Counting tests, not behaviours

Health — is the system sustainable?

KPI	How to measure	Healthy direction	Watch out for
Spec freshness / drift time	How long code and spec stay divergent after a hot‑fix	→ 0 (same‑day reconcile)	This is the leading indicator of framework decay
Spec coverage	% of shipped features with a current spec	→ 100%	"A spec exists" ≠ "the spec is correct"
MTTR (self‑healing loop)	Time from incident detected → fix shipped	↓	Fast MTTR that masks root cause
Blast‑radius incidents	Count of prod incidents on auth/payments/data	→ 0	One here outweighs many minor ones
Cost per shipped change	Token + platform spend ÷ changes	stable / ↓	Optimizing cost by under‑verifying

---

A minimal starter scorecard

You don't need all of the above. If you track four, track these — they cover flow, quality, and health and resist gaming as a set:

1. Cycle time (flow)
2. Regression escape rate (quality)
3. Spec drift time (health — the canary)
4. Blast‑radius incidents (the line you don't cross)

Speed metrics and the escape rate must be read together. Cycle time falling while escape rate rises means you're not faster — you're shipping bugs sooner.

---

Vanity metrics to ignore

Metric	Why it's noise for an SAE
Lines of code	Generation makes LOC free; more code is a liability, not output
Commit count / frequency	Measures activity, not value
Story points / velocity	Built to measure team coordination that no longer exists
Hours worked	The whole point is decoupling output from human hours
Number of features shipped	Ignores correctness and blast radius

---

Reality check. Any metric becomes a target you'll game — usually by quietly trading the metric you can't see for the one you can. The classic SAE trap: optimizing autonomy ratio ("look, no hand‑edits!") while regression escape rate climbs. Always pair a flow metric with a quality metric, and treat spec drift time as the canary — when it grows, the framework is decaying regardless of how good the other numbers look. For a solo operator these are personal instruments, not a performance review; their job is to make decay visible early, not to produce a dashboard for someone else.

---

See also: The End‑to‑End Lifecycle (which phase each metric measures) · Best Practices · Module 5: Project Navigation & Metrics.