Skip to content

Remove the transition_role=queue_storage_target requirement: re-resolve auto runtimes at the flip, or allow a quiesced-cluster flip #457

Description

@hardbyte

Problem

enter-mixed-transition requires a live runtime registered with transition_role=queue_storage_target. The role exists because a 0.6 runtime binds its execution engine once at startup: an auto runtime started pre-flip resolves canonical and can only downgrade to drain-only when the flip lands — it never starts executing queue-storage work. So the gate demands an explicit witness that someone will execute the new plane; storage_capability=queue_storage alone can't prove that.

In practice this forces every embedding application to plumb TransitionWorkerRole::QueueStorageTarget through its own config/deploy machinery for a one-time operation, then remove it (or leave a permanently misleading builder call). Verified on 0.6.5 that a live queue-storage-capable auto runtime is refused, and that a fully quiesced cluster (zero fresh heartbeats) is also refused.

Proposal 1 — auto runtimes re-resolve at the flip (removes the role)

Runtimes already observe transition state changes — that's how auto runtimes downgrade to canonical_drain_only after the flip. Instead of only downgrading, an auto runtime that observes mixed_transition could re-resolve and bring up its queue-storage runtime in place (internally close to "restart the client": a runtime started fresh during mixed_transition already resolves to queue storage, so the resolution logic exists; this is supervisor-level re-init, not new engine code).

The gate then relaxes to "≥1 live runtime with storage_capability=queue_storage", and TransitionWorkerRole can be deprecated for operators. This also fixes two adjacent sharp edges observed in a staged-transition rehearsal:

  • a pre-flip maintenance leader never promotes queue-storage deferred_jobs (its promotion path branches on its startup-resolved engine), so due queue-storage jobs sit unpromoted until leadership moves to a post-flip runtime;
  • the "roll all workers after finalize" step stops being load-bearing.

Proposal 2 — allow the flip on a quiesced cluster (much smaller)

If no runtime has a fresh heartbeat, allow enter-mixed-transition. With nothing live there is no fleet to strand: nothing can execute canonical work either, and any 0.6 runtime started after the flip resolves to queue storage at startup. This gives operators a role-free path: scale the fleet to zero, flip, (optionally migrate the deferred backlog, see #456), finalize, scale back up. For small/bursty deployments a brief quiesce window is often easier than coordinating a witness runtime.

Implementation wrinkle: the gate lives in the database (awa.storage_enter_mixed_transition, installed by v014), so relaxing it in SQL means a new migration on the 0.6 line — awkward with 0.7 owning v041–v043. Alternative: a CLI/awa-model-side path (e.g. enter-mixed-transition --quiesced) that performs its own liveness check and the state transition directly, refusing if any fresh heartbeat exists.

The two compose: quiesced clusters flip with no role today; live clusters keep the witness until proposal 1 lands, after which the role disappears entirely.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions