adapter: let the synchronous cut-over change the replication factor - #38174
Draft
aljoscha wants to merge 1 commit into
Draft
adapter: let the synchronous cut-over change the replication factor#38174aljoscha wants to merge 1 commit into
aljoscha wants to merge 1 commit into
Conversation
Changing a cluster's replication factor while a graceful reconfiguration is in progress is refused, because the record captures a target factor when written and the controller's cut-over writes it back to the realized config a tick later. A factor change applied independently in the meantime would be silently clobbered, with nothing in the ALTER's response to say so. That reasoning does not reach the synchronous cut-over. It folds the factor into the target it transacts and retires the record in the same transaction, and no consumer reads a retired record's target: `desired_replicas`, `target_hydrated` and the controller's own cut-over all sit behind `is_in_progress()`. There is no later cut-over left to clobber the write, so exempt it. The refusal cost the escape hatch its most useful shape. Forcing a wedged resize through while scaling down had to be two statements, and between them the cluster ran the old factor at the new size, the most expensive combination available, exactly while an operator is firefighting. `fold_reconfiguration_target` already computed the right answer for a re-targeted factor; only the guard made that branch unreachable. The escape hatch is the only exemption. A zero-timeout rollback still leaves a record in flight to settle on a tick, so it keeps the refusal, and the error hint now points at the cut-over as the third way out. The cut-over section gains the wedged case: the refusal still fires without a `WAIT`, and a combined size-and-factor cut-over lands both dimensions, converges the replica set, and settles the record `cancelled` because the record's own target factor was abandoned. The existing factor case ran with no record in flight, which is why this gap held. Also drops `Op::UpdateClusterReplicaConfig`. Deleting the staged reconfiguration machine removed its last producer, leaving the variant, its apply arm and its audit match arm dead.
aljoscha
force-pushed
the
aljoscha/cluster-legacy-07-cutover-factor
branch
from
August 12, 2026 08:47
b38afd8 to
3dfcb36
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #38112. Addresses both of @ggevay's review comments on #38103.
Let the synchronous cut-over change the replication factor
Changing the factor while a reconfiguration is in progress is refused, because
the record captures a target factor when written and the controller's cut-over
writes it back to the realized config a tick later. A factor change applied
independently in the meantime gets silently clobbered.
That reasoning doesn't reach the synchronous cut-over: it folds the factor into
the target it transacts and retires the record in the same transaction, and no
consumer reads a retired record's target (
desired_replicas,target_hydratedand the controller's own cut-over all sit behind
is_in_progress()). There's nolater cut-over left to do the clobbering, so it's exempt.
The refusal cost the escape hatch its most useful shape: forcing a wedged resize
through while scaling down had to be two statements, and in between the cluster
ran the old factor at the new size — the most expensive combination available,
exactly while someone is firefighting.
fold_reconfiguration_targetalreadycomputed the right answer for a re-targeted factor; only the guard made that
branch unreachable.
The exemption is narrow. A zero-timeout rollback still leaves a record in
flight to settle on a tick, so it keeps the refusal. The error hint now offers
the cut-over as a third way out, which is useful precisely because after this
change the hint only ever fires when you didn't ask for one.
Test coverage was the reason this held: the existing size-and-factor case runs
with no record in flight (its own comment says so). The cut-over section now also
wedges a record, asserts the refusal still fires without a
WAIT, then forces acombined size-and-factor cut-over and checks both dimensions land, the replica set
converges, and the record settles
cancelled— the record's own target factor wasabandoned, so
cancelledrather thanfinalizedis correct.Drop
Op::UpdateClusterReplicaConfigDeleting the staged reconfiguration machine removed its last producer. The
variant, its
transact.rsapply arm and itsddl.rsaudit match arm were dead.Confirmed no producers remain in
src/ortest/.Not in scope
@ggevay's comment prompted a wider question: should a factor change be allowed
while a reconfiguration is in progress generally, not just on the cut-over? The
factor is already a re-targetable dimension at record creation, and
mz_cluster_reconfigurations.changesplus theSHOW CLUSTERSsummary alreadyrender
replication factor to N— so the fold, the durable target and theuser-facing surface all support it, and only re-targeting an existing record is
refused.
That's a user-visible semantics change on a surface about to go GA, with at least
three coherent designs that differ in what the user pays, so it's tracked in
SQL-626 rather than smuggled in here. This PR is a bug fix with no semantic
change: it's correct under every one of those designs.