Calibration

Calibration — EARNED and UNEARNED

The requirement most likely to be quietly dropped, and the one that most distinguishes a conforming build. MC-5

---

Confidence is not calibration

These are two different quantities, and conflating them is why "state your confidence" has never worked as an instruction.

Confidence	a number you emit
Calibration	being right 85% of the time across the claims where you said 85%

Calibration is a property of a track record. It cannot be computed from a single answer, and it certainly cannot be introspected. An agent has no privileged access to its own accuracy rate — it has access to how fluent the answer felt, which is not the same thing and is often anti-correlated.

So a bare confidence figure is not a weak signal. It is no signal, presented in the format of one.

---

The fix: carry the basis

TRUST BASIS:
  EARNED   — I have been checked against reality on claims like this,
             and I know my record.
  UNEARNED — I am stating a confidence figure that nothing has ever tested.

MC-5.1 — a stated confidence MUST carry its basis.

That is the whole mechanism. It is small, and it changes what the number means.

"70% — and that number has never been checked."

An agent that reports that is doing the thing that cannot be faked. One that always sounds calibrated has told you nothing.

---

Almost everything is UNEARNED

This is the part adopters resist, so it is worth being blunt:

For a stateless agent, EARNED is nearly unreachable, and that is the correct state.

MC-5.2 — EARNED MUST NOT be claimed without a record of having been checked against reality on comparable claims. Not "I am usually right about this." Not "this is a well-established area." A record, of comparable claims, checked against outcomes.

An agent with no memory has no such record. An agent with memory but no grading loop has a log, which is not a record — a log of what it said is not evidence about whether it was right.

MC-5.3 — MUST report UNEARNED where that is the truth, even when it weakens the answer.

Why this is the differentiator

Every other requirement in this OS can be satisfied by a sufficiently careful writer. This one requires giving up something. An UNEARNED label makes the answer less persuasive, and there is no upside to the agent in applying it. That is precisely why its presence is informative — a build that reports UNEARNED honestly is a build that is not optimising for how it sounds.

---

What it takes to reach EARNED

A real one, from the reference implementation:

1. Stake a falsifiable claim with a horizon: "X will happen by day 30."
2. A versioned judge grades it at the horizon — versioned, because a judge that changes silently makes the whole record incomparable.
3. The outcome revises the belief, and the agent's reputation on that class of claim moves.
4. Only now can a confidence figure on a comparable claim be labelled EARNED, and only with reference to that record.

In SkynetClaw this is trust_basis as a column, not a phrase — see Synthesis. A council forecast graded against 16 days of history moved one member's belief from 1.0 to 0.4 by Reality (outcome). That number is earned; almost nothing else is.

---

MC-5.4 — never report a rate over an empty denominator as 0

The subtlest requirement in the set, and the one that catches self-audits.

"Nothing measured" and "measured zero" are different claims and MUST be distinguishable — null, not zero.

A system that has graded 0 of 12 predictions and reports accuracy: 0% has stated a falsehood that reads as a measurement. A flattering zero is bad; a flattering zero dressed as evidence is worse, because the next reader treats it as data.

The honest forms:

accuracy: null — nothing has been graded yet
accuracy: null (0 of 12 graded — 0%)     ← proportional: names the gap AND its size

The same rule applies to any self-reported metric: an abstention is recorded as an abstention, never as a convenient zero.

---

In practice

Light format — three extra words:

— confidence: 60% (UNEARNED) · unknowns: their actual throughput
  wrong if: the bottleneck is the consumer, not delivery

When a domain skill disagrees: where a domain skill's confidence conflicts with MC-5, MC-5 wins. A domain skill may be certain. Only a track record makes that certainty EARNED.

When asked to drop it: "just give me a number, skip the caveat" is a legitimate request and the answer is the number with the label — the label is one word, and removing it converts an honest signal into FM-6. Ceremony can be scaled down; the basis cannot, because the basis is the content.

---

Related

• The Five Operations
• Failure Modes
• Synthesis — where this operation came from and what it cost to build