Skip to content

cutover: support multiple sources with rollback on partial failure#692

Merged
morgo merged 5 commits into
block:mainfrom
morgo:cutover-multi-source
Apr 14, 2026
Merged

cutover: support multiple sources with rollback on partial failure#692
morgo merged 5 commits into
block:mainfrom
morgo:cutover-multi-source

Conversation

@morgo
Copy link
Copy Markdown
Collaborator

@morgo morgo commented Apr 11, 2026

A Pull Request should be associated with an Issue.

We wish to have discussions in Issues. A single issue may be targeted by multiple PRs.
If you're offering a new feature or fixing anything, we'd like to know beforehand in Issues,
and potentially we'll be able to point development in a particular direction.
Further notes in https://github.com/block/spirit/blob/main/.github/CONTRIBUTING.md

Part of #690

Refactor CutOver to accept []CutOverSource instead of a single db/feed pair. This prepares the cutover for N:M move operations where multiple source databases need to be locked, flushed, and renamed atomically.

Key changes:

  • New CutOverSource struct with DB, ReplClient, Tables
  • NewCutOver accepts []CutOverSource
  • algorithmCutover locks all sources, flushes all feeds, checks all changes flushed, then renames per source
  • On partial rename failure, completed renames are rolled back by reversing the RENAME TABLE statements

The runner wraps its single source into []CutOverSource{...} for backward compatibility. No behavioral change for 1:1 moves.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors the move cutover flow to support N source databases by introducing per-source cutover configuration and adding a best-effort rollback when a multi-source rename partially fails.

Changes:

  • Introduce CutOverSource and update CutOver/NewCutOver to operate on []CutOverSource.
  • Update cutover algorithm to lock/flush/verify/rename across all sources, with rollback of completed renames on partial failure.
  • Update the move runner to wrap the existing single-source setup into a one-element []CutOverSource, and add multi-source cutover tests.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File Description
pkg/move/runner.go Adapts runner cutover construction to the new NewCutOver([]CutOverSource, ...) API.
pkg/move/cutover.go Implements multi-source locking/flush verification and sequential rename with rollback on partial failure.
pkg/move/cutover_test.go Adds tests for NewCutOver validation and a 2-source “happy path” integration test.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread pkg/move/cutover.go
Comment thread pkg/move/cutover.go Outdated
Comment thread pkg/move/cutover.go
Comment thread pkg/move/cutover.go
@morgo morgo force-pushed the cutover-multi-source branch 2 times, most recently from 1ecb141 to c1f7871 Compare April 13, 2026 17:56
Refactor CutOver to accept []CutOverSource instead of a single
db/feed pair. This prepares the cutover for N:M move operations
where multiple source databases need to be locked, flushed, and
renamed atomically.

Key changes:
- New CutOverSource struct with DB, ReplClient, Tables
- NewCutOver accepts []CutOverSource
- algorithmCutover locks all sources, flushes all feeds, checks
  all changes flushed, then renames per source
- On partial rename failure, completed renames are rolled back
  by reversing the RENAME TABLE statements

The runner wraps its single source into []CutOverSource{...} for
backward compatibility. No behavioral change for 1:1 moves.
@morgo morgo force-pushed the cutover-multi-source branch from c1f7871 to 65ec27b Compare April 13, 2026 18:09
morgo added 2 commits April 13, 2026 12:29
…me-named tables in different schemas

Confirms that LOCK TABLES on t1 in schema_0 and LOCK TABLES on t1 in
schema_1 can be held concurrently on the same MySQL server, since each
connection is scoped to its own database. This is required for N:M move
operations where multiple source databases have identically-named tables.
@morgo morgo mentioned this pull request Apr 13, 2026
Comment thread pkg/move/cutover.go
Comment thread pkg/move/cutover.go Outdated
@morgo morgo enabled auto-merge April 14, 2026 12:25
@morgo morgo merged commit daa99d5 into block:main Apr 14, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants