Skip to content

adapter: remove ENABLE_CLUSTER_CONTROLLER and the legacy REFRESH scheduler - #38079

Merged
aljoscha merged 0 commit into
aljoscha/cluster-legacy-00-designfrom
aljoscha/cluster-legacy-01-scheduler
Aug 7, 2026
Merged

adapter: remove ENABLE_CLUSTER_CONTROLLER and the legacy REFRESH scheduler#38079
aljoscha merged 0 commit into
aljoscha/cluster-legacy-00-designfrom
aljoscha/cluster-legacy-01-scheduler

Conversation

@aljoscha

@aljoscha aljoscha commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Part 1 of 3 of the design for removing the legacy cluster paths.
Stacked on #38078 (the design doc). Parts 2 and 3 stack on this one.

Why

The cluster controller has been default-on since v26.29 and owns the replica
set of every user managed cluster in production. It still lives behind the
break-glass dyncfg ENABLE_CLUSTER_CONTROLLER, and that flag is the only thing
keeping the legacy REFRESH scheduler reachable: both of its entry points return
before doing any work while the gate is on, so it is a strict no-op in
production today.

Removing the gate means reverting to the legacy paths requires a binary
rollback. The gate has several releases of burn-in behind it, and the direct
reshape path (see part 3) remains as the operational escape hatch.

What lands

  • The ENABLE_CLUSTER_CONTROLLER dyncfg: definition, registration, its five
    read sites, and the sqllogictest binary's force-on entry.
  • src/adapter/src/coord/cluster_scheduling.rs wholesale:
    check_scheduling_policies, check_refresh_policy,
    handle_scheduling_decisions, and the SchedulingDecision /
    RefreshDecision types.
  • Its coordinator plumbing: Message::CheckSchedulingPolicies and
    Message::SchedulingDecisions with their handlers, the timer, the
    select-loop tick, and the cluster_scheduling_decisions state.
  • The metrics mz_check_scheduling_policies_seconds and
    mz_handle_scheduling_decisions_seconds (and their rows in the generated
    doc/user/data/metrics.yml).
  • The system var cluster_check_scheduling_policies_interval, whose sole
    consumer was the timer.
  • ReplicaCreateDropReason::ClusterScheduling, constructed only by the
    scheduler.

The persisted audit vocabulary stays: SchedulingDecisionsWithReasonsV2 and
friends are written by the controller's on-refresh path too
(refresh_window_decision_to_audit_log), and old events must remain decodable
regardless.

Tests

  • test/sqllogictest/mz_cluster_schedules.slt: the
    cluster_check_scheduling_policies_interval validation block goes with the
    var.
  • test/testdrive/cluster-controller.td: the cc_handoff* legacy-handoff
    scenarios and the cc_strand break-glass section are deleted (both are
    gate-off scenarios). The freeze-the-controller technique in the
    unmanaged-conversion refusal block is replaced by cranking
    cluster_controller_tick_interval up. Part 3 adds a replacement for
    cc_strand that exercises the direct cut-over escape hatch instead.
  • test/pg-cdc/cluster-graceful-reconfiguration.td: the explicit legacy
    foreground section goes, the controller section stays.
  • misc/python/materialize/mzcompose/__init__.py pins the flag only for
    versions below v26.38, so mixed-version runs against older binaries still
    exercise the same path. parallel_workload/action.py drops it from the
    do-not-flip list, and the two launchdarkly-flag-consistency allowlist entries
    go.

@aljoscha
aljoscha force-pushed the aljoscha/cluster-legacy-01-scheduler branch from 27b4877 to def651d Compare August 7, 2026 05:27
@aljoscha
aljoscha merged commit f283eea into main Aug 7, 2026
@aljoscha
aljoscha force-pushed the aljoscha/cluster-legacy-01-scheduler branch from def651d to f283eea Compare August 7, 2026 07:37
@aljoscha
aljoscha deleted the aljoscha/cluster-legacy-01-scheduler branch August 7, 2026 07:37
@aljoscha
aljoscha restored the aljoscha/cluster-legacy-01-scheduler branch August 7, 2026 07:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant