Deployment Ladder

The deployment ladder

The single most dangerous word in enterprise AI is "deploy."

---

Why the word is dangerous

"Deploy" implies a binary: off, then on.

That was roughly true for software whose failures were loud. It is not true for a system that can be running, responsive, cheap, and wrong, with nothing in the logs to say so.

A staged release is not caution. It is the only way to catch a failure at a rung where it is still cheap.

The five rungs

	Rung	What happens	What earns the next rung
1	Shadow	Processes real transactions. Outputs are captured and compared to the human's. Nothing is acted on	Accuracy within a defined band of the human baseline, sustained over a real period
2	Canary	Handles a small share of live volume. Humans handle the rest. Error rate and cost watched daily	Error rate inside the threshold, sustained, at real volume
3	Staged rollout	Volume rises in tranches, each one stable before the next	Each tranche stable for a defined period, with explicit sign-off per increase
4	Full production	Handles full volume within a defined scope. Humans handle exceptions and low-confidence cases	Sustained accuracy and unit cost at target
5	Sustainment	Golden dataset re-evaluated on a cadence. Business review quarterly. Risk charter renewed annually	No meaningful degradation in accuracy or cost efficiency

Rung 5 is not a resting state and it is the one most often skipped. A system at full production with no scheduled re-evaluation is not sustained, it is unattended.

What each rung actually tests

The rungs look like increasing volume. They are increasing exposure to reality, and each one admits a different kind of truth.

Shadow tests the agent against the work. It is the only rung where being wrong is free, which makes it the only rung where you will get an honest measurement.

Canary tests the agent against the operation. Handoffs, escalation, whether anyone notices the exceptions, whether the on-call path exists.

Staged rollout tests the organisation. Whether people trust it enough to stop double-checking, and whether the sign-off ritual is real or a formality.

Full production tests the economics. Unit cost at volume is a different number from unit cost in a pilot, usually in the wrong direction.

Sustainment tests time. Everything above was a measurement of a moment.

Skipping a rung does not save the time. It moves the discovery to a rung where it costs more.

Two examples, deliberately from different places

A back-office finance agent posting structured transactions: shadow against a month of real postings, canary on the lowest-risk counterparty, tranches by transaction type, full production within a scope that excludes anything unmatched.

A customer-facing support agent: shadow against closed tickets, canary on one topic with a low blast radius, tranches by topic rather than by percentage, and an exception path staffed before rung 2, not after.

The second one shows the trap. Percentage tranches are the wrong unit when risk varies by topic rather than by volume. Ask what the tranche is actually a tranche of.

The failure this ladder is built against

Software crashes loudly. Agents fail silently.

An agent can keep running, without errors, while its outputs slowly stop matching what the work requires, because an upstream policy changed or a model was updated underneath it.

That is why every rung's exit criterion is a sustained measurement rather than a successful run. A single good day is not evidence of anything, and the ladder is designed so that no rung can be cleared by a demonstration.

The instrument for rung 5 is the three drifts.

Three rules that make the ladder work

Demotion exists. A ladder with no way down is a ratchet. Something at rung 4 after an unlogged incident is at rung 4 for good.

The exit criterion is written before the rung is occupied. Written afterwards, it is a description of what happened.

Rollback is tested, not documented. A tested rollback is a control. A documented one is a plan you will execute for the first time under pressure.

_{Adapted from the AI CoE and Agent Factory Playbook, based on The Augmented Enterprise framework. Vertical specifics have been generalised.}

_{Back to Home · The three proof gates · The three drifts}