Skip to content

Move table mode for gh-ost #1681

@danieljoos

Description

@danieljoos

This document specifies the work needed inside gh-ost to support a new move table mode, in which one or more tables are copied from a source MySQL cluster to a target MySQL cluster, with a cooperative cutover protocol coordinated via hooks.

Scope

In scope:

  • One or more tables in a single gh-ost run, sharing one source binlog stream.
  • GTID-only operation.
  • gh-ost issues the source RENAME TABLE itself; ACL changes and feature-flag flips are delegated to hook scripts.

Explicitly deferred or out of scope:

  • Schema transformation (--alter) combined with move.
  • File+position binlog drain (non-GTID).
  • Inline checksumming or target-binlog monitoring (use pt-table-checksum externally).
  • Revert / rollback of a completed move.
  • Source-side grant cleanup.

Design

Move table mode is enabled by a single new flag --move-tables=t1,t2,.... The mode reuses the existing inspector, binlog streamer, chunk iterator, throttler, hook executor, and checkpoint mechanism, while introducing a second database connection (the target), per-table runtime state, and a cooperative cutover protocol driven by gh-ost.

Architecture overview

┌──────────────────────┐                    ┌──────────────────────┐
│     Source MySQL     │                    │     Target MySQL     │
│                      │                    │                      │
│  ┌─────────┐         │                    │  ┌─────────┐         │
│  │ table_A │◄── Inspector reads ──┐       │  │ table_A │         │
│  │ table_B │                      │       │  │ table_B │         │
│  └─────────┘         │            │       │  └─────────┘         │
│                      │            │       │       ▲              │
│  Binlog ────────────────► EventsStreamer  │       │              │
│                      │            │       │   Applier writes ────┘
│                      │        gh-ost      │                      │
│                      │       process      │  ┌─────────┐         │
│                      │            │       │  │  _ghk    │        │
│                      │            └───────│──│checkpoint│        │
│                      │                    │  └─────────┘         │
└──────────────────────┘                    └──────────────────────┘

Key principle: limit changes to the hot code path (eg. binlog streamer). All move-table-specific logic lives in higher layers — the migrator (cutover orchestration and multi-table iteration) and the applier (where it connects and what tables it writes to).

Connection model

Connection Standard mode Move table mode
Inspector reads from source reads from source (unchanged)
Binlog streamer reads from source reads from source (unchanged)
Applier writes to source-side master writes to target master
Cutover issues RENAME on source issues RENAME on source (short-lived, reuses inspector connection config)

Cutover protocol

The read cutover is the application repoints its reads to the target while writes still flow to the source. The write cutover is the actual "cutover", in which gh-ost issues a single atomic RENAME TABLE on the source, captures @@gtid_executed as the drain marker, waits until the applier has reached that GTID, and fires on-success so the hook can unlock the target write user and flip the application's write feature flag.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions