Skip to content

feat(cli,core): gate the DE parallel router on the canary and ramp to 5% - #3095

Closed
vanceingalls wants to merge 1 commit into
mainfrom
canary-gate-de-router
Closed

feat(cli,core): gate the DE parallel router on the canary and ramp to 5%#3095
vanceingalls wants to merge 1 commit into
mainfrom
canary-gate-de-router

Conversation

@vanceingalls

Copy link
Copy Markdown
Collaborator

What

Router trial arming is now gated on the de-parallel-router canary, and the registry moves 0 → 5%.

Previously every eligible install armed the trial. That produced the soak — 1,466 installs, with 154 of 155 reverting installs stopping at exactly one — but it is also an unbounded blast radius the moment the router regresses. The precedent is 0.7.60–0.7.64, where every unclamped render reverted for five consecutive releases, silently.

The exposed fraction is now an explicit, revertible number. Setting the percentage back to 0 switches the rollout off for everyone with no code change.

The bucketer was validated first

Calibration at n = 13,547 installs:

check result
calibration-10 9.62% (target 10%) ✓
calibration-50 49.76% (target 50%) ✓
independence passes
forced_on/forced_off 0 — overrides ruled out
CI population self-identifying via excluded
sustained cohort flips 0.10%

That 0.10% is an order of magnitude below this feature's own ~2.79% revert rate, so cohort instability cannot meaningfully corrupt the ramp read. It has also held steady across three reads (0.14% → 0.12% → 0.10%), so it is a stable edge rather than something growing.

Replaces a test that could never pass

canary.test.ts pinned the percentage to 0 with the note "ramp only alongside the per-install circuit breaker". But pinning 0 blocks the ramp permanently and never checks the wiring it names.

It now asserts the wiring directly: a non-zero percentage is allowed only while the CLI render path gates on this canary and still consults the per-install breaker. Mutation-tested — removing the gate fails it.

What to watch at each step

Split revert rate by cpu_count and is_docker. Those two segments (~12% of eligible renders between them) had near-zero trial coverage, and worker-count-dependent failure is exactly how 0.7.60–0.7.64 broke.

Useful baseline from telemetry: routed renders fail at 2–3.4%, stable across versions.

Relationship to #2840

This supersedes #2840's approach. That PR flips to unconditional default-on (unset = ON); this ramps through the registry instead, which is the safer shape for the same goal. #2840 can close once this lands, or be reworked to drop the polarity change and keep only its kill-switch parsing fixes.

Verification

core 1674, cli 2486 passing. Lint clean. Both the gate and the wiring assertion mutation-tested.

One pre-existing failure on main, not from this branch: audioPadTrim.realmedia times out at 5s — reproduced on clean origin/main.

🤖 Generated with Claude Code

Every eligible install used to arm the router trial. That produced the soak
(1,466 installs, 154 of 155 reverting installs stopping at exactly one), but
it is also an unbounded blast radius the moment the router regresses —
0.7.60-0.7.64 is the precedent, where every unclamped render reverted for
five consecutive releases, silently.

Arming is now gated on the de-parallel-router canary, and the registry moves
0 -> 5. The exposed fraction becomes an explicit, revertible number: setting
the percentage back to 0 switches the rollout off for everyone with no code
change.

The bucketer was validated first. Calibration at n=13,547: accuracy
9.62%/49.76% against 10%/50% targets, overrides and the CI population both
attributable now that the reason ships, and sustained cohort flips at 0.10% —
an order of magnitude below this feature's own ~2.79% revert rate, so cohort
noise cannot corrupt the read.

Replaces the registry test that pinned the percentage to 0. Its stated intent
was 'ramp only alongside the circuit breaker', but pinning 0 blocks the ramp
forever and never checks the wiring it names. It now asserts the wiring
directly: a non-zero percentage is allowed only while the CLI render path
gates on this canary and still consults the per-install breaker.

At each ramp step, split revert rate by cpu_count and is_docker — the two
segments the opt-in trial never covered.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vanceingalls

Copy link
Copy Markdown
Collaborator Author

Closing — superseded by #2840, which is the right shape for this.

This PR gated the existing trial on the canary. That was wrong in two ways:

  1. It reduced exposure rather than governing it. About 10.85% of rendering installs already route today; a 5% canary would roughly halve that, shrinking the soak while adding no safety the per-install breaker doesn't already provide.
  2. My framing of "unbounded blast radius" was wrong. Today's exposure is bounded — by the breaker latch, the 25-render cap, and the telemetry preconditions. The ~11% is emergent from eligibility rules, not a designed number, but it is not unbounded.

What actually needs ramping is the expansion: #2840 removes the 25-render cap and the telemetry couplings, taking routed renders from ~6% of eligible toward 100%. That is the ~17x jump, and that is what the canary should gate — enrolled installs get uncapped default-on, everyone else keeps today's behavior unchanged, and 0% means nothing changes at all.

The one piece worth keeping from here is the registry test change: the old test pinned de-parallel-router to 0% with the note "ramp only alongside the per-install circuit breaker", which blocks the ramp permanently and never checks the wiring it names. I will carry the wiring assertion over to #2840.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant