adapter: remove ENABLE_CLUSTER_CONTROLLER and the legacy REFRESH scheduler - #38079
Merged
aljoscha merged 0 commit intoAug 7, 2026
Merged
Conversation
aljoscha
force-pushed
the
aljoscha/cluster-legacy-01-scheduler
branch
from
August 7, 2026 05:27
27b4877 to
def651d
Compare
aljoscha
force-pushed
the
aljoscha/cluster-legacy-01-scheduler
branch
from
August 7, 2026 07:37
def651d to
f283eea
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part 1 of 3 of the design for removing the legacy cluster paths.
Stacked on #38078 (the design doc). Parts 2 and 3 stack on this one.
Why
The cluster controller has been default-on since v26.29 and owns the replica
set of every user managed cluster in production. It still lives behind the
break-glass dyncfg
ENABLE_CLUSTER_CONTROLLER, and that flag is the only thingkeeping the legacy REFRESH scheduler reachable: both of its entry points return
before doing any work while the gate is on, so it is a strict no-op in
production today.
Removing the gate means reverting to the legacy paths requires a binary
rollback. The gate has several releases of burn-in behind it, and the direct
reshape path (see part 3) remains as the operational escape hatch.
What lands
ENABLE_CLUSTER_CONTROLLERdyncfg: definition, registration, its fiveread sites, and the sqllogictest binary's force-on entry.
src/adapter/src/coord/cluster_scheduling.rswholesale:check_scheduling_policies,check_refresh_policy,handle_scheduling_decisions, and theSchedulingDecision/RefreshDecisiontypes.Message::CheckSchedulingPoliciesandMessage::SchedulingDecisionswith their handlers, the timer, theselect-loop tick, and the
cluster_scheduling_decisionsstate.mz_check_scheduling_policies_secondsandmz_handle_scheduling_decisions_seconds(and their rows in the generateddoc/user/data/metrics.yml).cluster_check_scheduling_policies_interval, whose soleconsumer was the timer.
ReplicaCreateDropReason::ClusterScheduling, constructed only by thescheduler.
The persisted audit vocabulary stays:
SchedulingDecisionsWithReasonsV2andfriends are written by the controller's on-refresh path too
(
refresh_window_decision_to_audit_log), and old events must remain decodableregardless.
Tests
test/sqllogictest/mz_cluster_schedules.slt: thecluster_check_scheduling_policies_intervalvalidation block goes with thevar.
test/testdrive/cluster-controller.td: thecc_handoff*legacy-handoffscenarios and the
cc_strandbreak-glass section are deleted (both aregate-off scenarios). The freeze-the-controller technique in the
unmanaged-conversion refusal block is replaced by cranking
cluster_controller_tick_intervalup. Part 3 adds a replacement forcc_strandthat exercises the direct cut-over escape hatch instead.test/pg-cdc/cluster-graceful-reconfiguration.td: the explicit legacyforeground section goes, the controller section stays.
misc/python/materialize/mzcompose/__init__.pypins the flag only forversions below v26.38, so mixed-version runs against older binaries still
exercise the same path.
parallel_workload/action.pydrops it from thedo-not-flip list, and the two launchdarkly-flag-consistency allowlist entries
go.