Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial stubs for resharding driver #4355

Merged
merged 10 commits into from
May 30, 2024
Merged

Initial stubs for resharding driver #4355

merged 10 commits into from
May 30, 2024

Conversation

timvisee
Copy link
Member

@timvisee timvisee commented May 30, 2024

Tracked in: #4213

Lay out the initial stubs for the logic driving the whole resharding process.

All Submissions:

  • Contributions should target the dev branch. Did you create your branch from dev?
  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?

New Feature Submissions:

  1. Does your submission pass tests?
  2. Have you formatted your code locally using cargo +nightly fmt --all command prior to submission?
  3. Have you checked your code using cargo clippy --all --all-features command?

@ffuugoo ffuugoo mentioned this pull request May 30, 2024
76 tasks
@timvisee timvisee requested a review from ffuugoo May 30, 2024 09:26
Copy link
Contributor

@ffuugoo ffuugoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like we don't neccessary need ReshardTask. ReshardKey + ShardHolder::resharding_state should be enough, but LGTM otherwise.

lib/collection/src/collection/resharding.rs Outdated Show resolved Hide resolved
@timvisee
Copy link
Member Author

I feel like we don't neccessary need ReshardTask. ReshardKey + ShardHolder::resharding_state should be enough, but LGTM otherwise.

You might be right. My idea was that we might add additional information such as the desired shard distribution.

I'll remove it for now. We can add it back later once we think it'll be useful again.

@timvisee timvisee merged commit 22d7581 into dev May 30, 2024
17 checks passed
@timvisee timvisee deleted the resharding-driver-stubs branch May 30, 2024 14:37
timvisee added a commit that referenced this pull request Jun 11, 2024
* Initial stubs for resharding driver

* Add basic steps in resharding driver

* Link Notion document

* Clean up start function

* Describe difference between reshard key and task

* Report resharding abort/finish to consensus

* Add driver methods for remaining stages

* Fix typo

* Only start driver on the peer that is responsible for driving it

* Remove ReshardTask, repurpose ReshardKey for that
@timvisee timvisee added this to the v1.11: resharding milestone Jul 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants