Repository navigation
North Star Business Case
sergio velderrain edited this page Nov 17, 2026
·
1 revision
This is the public summary of the north star (VITechnologies/projectbuddy#328): one measured lbabus wiki change, an improvement to the standard, and a rerun. Figures are relative. The full case, with absolute figures and the stakeholder deck, is private.
Under standard @4, implementing the change cost 17.6% less AI to acceptance than the mean of two control runs
under @2, in 12.6% less wall-clock time. The defect the improvement targeted did not recur. It was missing
scope on the test pages of allocated requirements, and it caused every control-run gate resume.
| Run | Standard | AI cost ÷ control mean | Wall-clock ÷ control mean | Gate resumes |
|---|---|---|---|---|
| A | @2 |
1.13 | 1.16 | 1 |
| A2 | @2 |
0.87 | 0.84 | 2 |
| B | @3 |
1.12 | 1.48 | 0 |
| B2 | @4 |
0.82 | 0.87 | 1 |
- Pre-registered in VITechnologies/projectbuddy#337: the same baseline commit, a frozen prompt, the same model, and a frozen acceptance gate (the reference checker, plus an independent review).
- Unattended: each run is a scripted Copilot CLI session with no operator and no route to any remote. A run that fails its gate is resumed with the findings until it passes. Its cost is the total.
- Metered in AI Units, the unit the quota is billed in. Each run's AI Units come from its own session records, and the account quota measured around each step agreed with them, to between −0.04% and +8.8% (VITechnologies/projectbuddy#339).
-
A rule that is ambiguous can cost more than it saves. Standard
@3said that a test page "names the member that verifies" a condition it leaves unverified, and a run read that as "prove it". It audited another repository and cost 12% more than the control. Standard@4takes the verifying member from the allocating specification instead. That fixed the problem and paid back its own cost after about 2.5 changes. - Operator time is the largest lever. It was 70.7% of the workstation's baseline cost, and the measured runs needed none of it.
- Caveat: the two control runs differ from each other by 25%. With one run per treatment, the 17.6% is not statistically significant. The effect on the targeted defect was the clearest result.
- Adopt
@4across the ecosystem. - Iteration 3: an observation claims only what its stimulus shows.
- At least 3 runs per arm, for significance.