Confidence ladder

Reading the confidence ladder

Every number OikosBot emits carries a confidence label. This is the single most important thing to understand about the tool, because the labels are not decoration — they govern what the tool is allowed to do with the number.

The four levels

Level	Means	May block a build?
Measured	Came directly from an API or an instrument. Not modelled.	Yes
Calibrated	A coefficient applied over a measurement.	Yes
Estimated	Derived from a heuristic or a declared assumption.	No
Unknown	Provenance not established.	No

The rule is enforced in code, not by convention: assess() marks a verdict actionable only when every objective driving it ranks Calibrated or better.

Where each quantity sits, and why

For the estate pipeline:

Quantity	Source	Level	Why not higher
Wall-clock minutes	GitHub API, directly	Measured	—
Money	minutes × published per-minute rate	Calibrated	The rate is declared, not observed per-run
Energy	minutes × runner power × PUE	Calibrated	Power and PUE are coefficients over a real measurement
Carbon	energy × grid intensity	Estimated	GitHub does not expose which region a runner ran in. Intensity therefore rests on a declared assumption

That last row is the point of the whole mechanism. Carbon is the number a sustainability tool is most tempted to lead with, and it is the weakest claim here — so it is labelled as the weakest claim here, and it is not permitted to fail anyone's build.

The inversion is deliberate: the economic axes are the strong ones. Money and time come from the API. That is what "treating the economic as seriously as the ecological" means in practice.

The per-file path is weaker, and says so

The per-file analyser (analyze / check / report / compare) emits only Estimated. Its resource figures are derived from a single AST node count, so --check cannot block a merge — by design, not by accident.

When you ask it to enforce something it cannot justify, it refuses loudly:

::warning::--check requested but NOT enforced: the objectives driving this
Regression verdict are [Estimated], and only Measured or Calibrated inputs
may block.

An earlier version exited 0 silently in exactly this situation. That was the more dangerous behaviour: a gate that cannot fail, reporting success, is worse than no gate — people learn to trust the green light.

Wiring real calibration is tracked as issue #48.

How to use the labels

• Do not average across levels. A Measured and an Estimated number are not the same kind of object; combining them into one score launders the uncertainty.
• Gate on Measured/Calibrated only. If you want a blocking check today, build it on minutes and money, not carbon.
• Treat Estimated as directional. Useful for ranking and for spotting outliers; not defensible as an absolute figure in a report to anyone else.

See also

• What is owed — the honest account of what is and is not earned
• The estate pipeline — where the Measured data comes from