Skip to content
This repository has been archived by the owner on Jun 23, 2022. It is now read-only.

split: parent replica create child replica #291

Merged
merged 22 commits into from
Aug 21, 2019

Conversation

hycdong
Copy link
Contributor

@hycdong hycdong commented Aug 8, 2019

Simple partition split process

  1. meta receives client partition split request, and change partition count split: meta start partition split #286
  2. replica notices partition count changed during on_config_sync
  3. parent partition create child partition
  4. child partition async learn mutations and private logs from parent
  5. all child partitions in group finish learn process, parent will stop read and write
  6. meta server register child partitions
  7. child partition active, and parent recover read and write

More partition split discussion in issue #69
This pr solves the third step of partition split, which is bold in process description.

Main logic

  • Function on_add_child - parent replica create child replica
    • check replica status
    • check ballot ( we assume that partition split should stop if parent ballot changed)
    • check child_gpid validation
  • Function create_child_replica - create child instance
    • get replica by gpid and create a new replica instance if not found
  • Function init_child_replica - child replica initialized

More explanations

  • Update replica::newr and replica_stub::get_replica_dir, create replica_stub::get_child_dir
    • we should grantee child replica store data on the same disk as parent dose
    • for example, current partition count is 4, parent replica data_dir is 'ssd2/1.1.pegasus/reps', child replica should be 'ssd2/1.5.pegasus/reps'
    • as a result, I refactor function get_replica_dir and add function get_child_dir to support creating replica under specific dir rather than allocating by _fs_manager.
  • Add new partition_status: PS_PARTITION_SPLIT and its context function
    • child replica during partition split will be PS_PARTITION_SPLIT
    • partition_split_context in the pr is incomplete, more varieties and functions will be add in following pull request
  • Add helper function split_replica_exec to simplify code
  • on_add_child is called during on_config_sync, not in this pr

Tests

  • Add unit test for function on_add_child
  • Unit test for function init_child_replica is NOT added in this pr, because current init_child_replica is incomplete

src/dist/replication/lib/replica.cpp Outdated Show resolved Hide resolved
src/dist/replication/lib/replica.cpp Outdated Show resolved Hide resolved
src/dist/replication/lib/replica.h Outdated Show resolved Hide resolved
src/dist/replication/lib/replica_stub.h Outdated Show resolved Hide resolved
src/dist/replication/lib/replica_stub.cpp Outdated Show resolved Hide resolved
src/dist/replication/lib/replica_stub.h Outdated Show resolved Hide resolved
src/dist/replication/lib/replica_stub.cpp Outdated Show resolved Hide resolved
src/dist/replication/lib/replica_stub.cpp Outdated Show resolved Hide resolved
src/dist/replication/lib/replica_stub.cpp Outdated Show resolved Hide resolved
src/dist/replication/lib/replica_stub.cpp Outdated Show resolved Hide resolved
src/dist/replication/lib/replica_stub.cpp Outdated Show resolved Hide resolved
src/dist/replication/lib/replica_split.cpp Show resolved Hide resolved
src/dist/replication/lib/replica_split.cpp Outdated Show resolved Hide resolved
src/dist/replication/lib/replica.h Show resolved Hide resolved
src/dist/replication/lib/replica_split.cpp Outdated Show resolved Hide resolved
src/dist/replication/lib/replica_stub.cpp Outdated Show resolved Hide resolved
src/dist/replication/lib/replica.h Outdated Show resolved Hide resolved
src/dist/replication/lib/replica_stub.cpp Outdated Show resolved Hide resolved
src/dist/replication/lib/replica_stub.cpp Outdated Show resolved Hide resolved
src/dist/replication/lib/replica_context.h Show resolved Hide resolved
@vagetablechicken vagetablechicken merged commit ca6dd5a into XiaoMi:master Aug 21, 2019
@hycdong hycdong deleted the replica_add_child branch April 22, 2020 01:31
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants