Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor meta part table #3689

Closed
critical27 opened this issue Jan 11, 2022 · 1 comment · Fixed by #3771
Closed

Refactor meta part table #3689

critical27 opened this issue Jan 11, 2022 · 1 comment · Fixed by #3771
Assignees
Labels
priority/hi-pri Priority: high type/enhancement Type: make the code neat or more efficient
Milestone

Comments

@critical27
Copy link
Contributor

We only save who has given partition in meta for now, this works fine in normal cases. But during balance data, the part allocation will be modified once, for example, part 1 is on [A, B, C] and we want to replace A with D:

...
open part 1 on D
add D as learner
add D as peer
remove A from peer
modify meta from [ABC] to [BCD]

Actually, we need to handle the meta not only in last step, add learner/add peer/remove peer also need to handle it as well. What we need to do is that add a flag to indicate whether a host is learner or not.

Some places need to be modified, almost everywhere we use the part table in meta

  1. create space: all part is not learner
  2. balance data:
    • when we add learner, add D into the kv as learner
    • when we promote D as normal replica, modify D as normal one in kv
    • when we remove A from raft, modify the kv as well
  3. start up process, since the meta table is modified, when starting up, we need to know my raft peers, some of then maybe learner, we need to use all data on meta.
@critical27 critical27 added type/enhancement Type: make the code neat or more efficient priority/hi-pri Priority: high labels Jan 11, 2022
@critical27 critical27 self-assigned this Jan 11, 2022
@Sophie-Xie Sophie-Xie added this to the v3.0.0 milestone Jan 11, 2022
@Sophie-Xie Sophie-Xie added the type/bug Type: something is unexpected label Jan 13, 2022
@critical27
Copy link
Contributor Author

After talked with @pengweisong offline, we could use a simple method to address this issue. Save the peer info in storage local. This won't involve too much changes in meta.

@pengweisong pengweisong mentioned this issue Jan 20, 2022
11 tasks
@Sophie-Xie Sophie-Xie modified the milestones: v3.0.0, v3.1.0 Jan 20, 2022
@Sophie-Xie Sophie-Xie linked a pull request Feb 25, 2022 that will close this issue
11 tasks
@Sophie-Xie Sophie-Xie removed the type/bug Type: something is unexpected label Feb 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority/hi-pri Priority: high type/enhancement Type: make the code neat or more efficient
Projects
No open projects
Status: Done
Development

Successfully merging a pull request may close this issue.

3 participants