canary #668 increment 3: opt-in human confirmation at ring1→stable (require_confirmation, dev-lead)

[don-petry]

[don-petry]

Dev-Lead Implementation Plan

Issue: #677 — canary #668 increment 3: opt-in human confirmation at ring1→stable (require_confirmation, dev-lead)

Scope

Implement Layer 3 of the #668 design: an opt-in, evidence-carrying human go/no-go at the ring1->stable boundary for dev-lead only, via a new AWAITING_CONFIRMATION gate state that the scheduled promote-all never auto-advances and that a deliberate promote <agent> --confirm dispatch clears. Engine-only; every other agent keeps lights-out promotion (default-off, byte-identical).

Design (per the #668 delivery-plan step 3)

• Pure core unchanged (scripts/lib/canary-rollout.sh): the overlay lives in the orchestrator.
• AWAITING_CONFIRMATION = reliability verdict is PROMOTE and the frontier transition sets require_confirmation: true and no confirmation is present. The confirmation is the dispatch (--confirm) — no new state store, consistent with "channel tags ARE the state".
• --confirm is not --override: it advances only an otherwise-PROMOTE (reliability-clean) AWAITING_CONFIRMATION state; it can never bypass a BLOCKED gate.

Implementation Checklist

• standards/canary-rings.json — add require_confirmation: true to dev-lead gate.transitions.ring1->stable + a _confirm_note
• scripts/canary-rollout.sh _frontier_state — overlay PROMOTE → AWAITING_CONFIRMATION when the transition requires confirmation
• scripts/canary-rollout.sh cmd_promote — add --confirm flag; advance AWAITING_CONFIRMATION only with --confirm (or --override); informative hold message otherwise
• scripts/canary-rollout.sh cmd_evaluate — report the new state
• scripts/canary-rollout.sh — add _confirm_body (evidence-carrying: compare <stable>...<cand> diff link, run stats, the promote --confirm instruction)
• scripts/canary-rollout.sh cmd_sync_issues — upsert/auto-close a canary-confirm:<agent> issue (label canary-confirm + needs-human); show the state in the fleet dashboard
• scripts/canary-rollout.sh — usage/help strings for --confirm
• .github/workflows/canary-rollout.yml — add a confirm boolean workflow_dispatch input, threaded into the promote args

Tests to Write (TDD, tests/canary_rollout.bats)

• registry: dev-lead ring1->stable.require_confirmation == true; default-absent for opted-out agents
• _frontier_state/evaluate: dev-lead at ring1->stable, clean + dwell + sample met → AWAITING_CONFIRMATION (not PROMOTE)
• promote without --confirm on AWAITING_CONFIRMATION → holds (no tag move), prints the confirm instruction
• promote --confirm --dry-run → advances (shows the gh-api move)
• promote --confirm on a BLOCKED+REGRESSION frontier → still held (proves --confirm ≠ --override)
• sync-issues → opens a canary-confirm issue with the compare link + needs-human; dashboard shows the state
• sync-issues auto-closes a stale confirm issue once the agent is no longer awaiting
• _confirm_body renders the compare diff link + the promote --confirm guidance

Test Command

bats tests/canary_rollout.bats (plus bash .dev-lead/scripts/dev-lead-lint.sh for shellcheck)

[don-petry]

✅ Issue #677 complete — canary #668 increment 3: opt-in human confirmation at ring1->stable

Layer 3 of the #668 design: an opt-in human go/no-go at a single ring boundary for a correctness-sensitive agent, implemented via a new AWAITING_CONFIRMATION gate state. dev-lead only initially; everything default-off and byte-identical for opted-out agents.

Plan Execution

• Phase 1 — Scope & Plan (plan comment posted before touching source)
• Phase 2 — Test-First TDD (8 new failing tests written & confirmed red first)
• Phase 3 — Implement (orchestrator overlay + registry knob + workflow input)
• Phase 4 — Verify (full suite green + dev-lead-lint.sh exit 0)
• Phase 5 — Rubber Duck (git diff self-review, all four files)
• Phase 6 — Completion report (this comment)

Semantics delivered

• New state AWAITING_CONFIRMATION. When _frontier_state computes reliability PROMOTE and the transition sets require_confirmation: true, the frontier holds in AWAITING_CONFIRMATION instead of advancing. Reliability still clears next/ring0/ring1 unattended — only ring1->stable gates.
• The confirmation IS the dispatch — no persistent state store. A deliberate promote <agent> --confirm (Actions-audit-logged) advances it; the scheduled promote-all never sets --confirm, so it holds indefinitely without a human.
• --confirm is NOT --override. It advances only a reliability-clean AWAITING_CONFIRMATION state and can never bypass a BLOCKED gate (verified by test 131). The overlay only fires from PROMOTE, and the BLOCKED+REGRESSION/SUSPECT guard still requires override.
• Evidence-carrying issue. sync-issues files/updates a marker-keyed canary-confirm:<agent> issue (label canary-confirm, --add-label needs-human) carrying a compare/<prior>...<cand> diff link + run stats, and auto-closes it once the agent is no longer awaiting (promote or candidate change).

Test Results

1..134
134 ok / 0 not ok

All 8 new increment-3 tests (127–134) green; no existing test modified or skipped. dev-lead-lint.sh (shellcheck --severity=warning -x on 21 scripts + frontmatter + CODEOWNERS): all checks passed, exit 0. JSON/YAML valid.

New tests:

• 127 — dev-lead ring1->stable opts into require_confirmation; opted-out agents default-off
• 128 — reliability PROMOTE holds as AWAITING_CONFIRMATION
• 129 — does NOT advance without --confirm (scheduled sweep holds)
• 130 — promote --confirm advances a reliability-clean frontier
• 131 — --confirm cannot advance a BLOCKED gate
• 132 — _confirm_body renders compare diff link + --confirm instruction
• 133 — sync-issues opens a canary-confirm issue
• 134 — sync-issues auto-closes a stale confirm issue

Files Changed

 .github/workflows/canary-rollout.yml |   9 ++
 scripts/canary-rollout.sh            | 123 +++++++++++++++++++++++--
 standards/canary-rings.json          |   4 +-
 tests/canary_rollout.bats            | 170 +++++++++++++++++++++++++++++++++++
 4 files changed, 299 insertions(+), 7 deletions(-)

• standards/canary-rings.json — dev-lead ring1->stable gains require_confirmation: true + _confirm_note. All other agents null (default-off).
• scripts/canary-rollout.sh — _frontier_state overlay; cmd_evaluate notice; cmd_promote --confirm handling + hold message; new _confirm_body; cmd_sync_issues confirm-issue upsert/close + canary-confirm label; dashboard legend; CLI usage/header.
• .github/workflows/canary-rollout.yml — confirm boolean dispatch input → CONFIRM env → --confirm arg in the promote case.

Notes

• Pure decision core (scripts/lib/canary-rollout.sh) unchanged per the design mandate — the entire Layer 3 overlay lives in the orchestrator.
• Default-off invariant held: adding the knob to dev-lead breaks no existing test; the positional _frontier_state output stays single-token so downstream parsing is unaffected.
• Per task constraints, no commit/push/PR — CI handles all git operations.