Skip to content

split batches for RawKV batch put / delete requests #501

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 8 commits into from
Aug 25, 2025
Merged

split batches for RawKV batch put / delete requests #501

merged 8 commits into from
Aug 25, 2025

Conversation

lance6716
Copy link
Contributor

@lance6716 lance6716 commented Aug 25, 2025 •
edited
Loading

Without the code fix, the new test will fail with

RegionError(Error { message: "raft entry is too large, region 8, entry size 107438290" ...

see https://github.com/tikv/client-rust/actions/runs/17204354874/job/48801577491?pr=502

@lance6716
split batches for RawKV requests
6b34ce7 
Signed-off-by: lance6716 <lance6716@gmail.com>
@ti-chi-bot ti-chi-bot bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. dco-signoff: yes Indicates the PR's author has signed the dco. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Aug 25, 2025
@lance6716
add test
3b0102e 
Signed-off-by: lance6716 <lance6716@gmail.com>
@lance6716
format
e78d803 
Signed-off-by: lance6716 <lance6716@gmail.com>
@lance6716
cargo fmt
d4fcc7a 
Signed-off-by: lance6716 <lance6716@gmail.com>
@lance6716 lance6716 changed the title [WIP] split batches for RawKV requests split batches for RawKV batch put requests Aug 25, 2025
@ti-chi-bot ti-chi-bot bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 25, 2025
@lance6716
remove unused change
669774e 
Signed-off-by: lance6716 <lance6716@gmail.com>
@ti-chi-bot ti-chi-bot bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved labels Aug 25, 2025
lance6716
lance6716 commented Aug 25, 2025
View reviewed changes
src/raw/requests.rs
self.ttls = ttls;
}

fn clone_then_apply_shard(&self, shard: Self::Shard) -> Self
Copy link
Contributor Author

@lance6716 lance6716 Aug 25, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This skips cloning the large "batch put" payload in self.pairs when splits into shards' requests

@ti-chi-bot ti-chi-bot
Copy link

ti-chi-bot bot commented Aug 25, 2025

@sleepymole: adding LGTM is restricted to approvers and reviewers in OWNERS files.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

pingyu
pingyu reviewed Aug 25, 2025
View reviewed changes
Copy link
Collaborator

@pingyu pingyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest LGTM.

@lance6716
Copy link
Contributor Author

/hold

@ti-chi-bot ti-chi-bot bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 25, 2025
@lance6716
also split batch delete
c546874 
Signed-off-by: lance6716 <lance6716@gmail.com>
@lance6716 lance6716 changed the title split batches for RawKV batch put requests split batches for RawKV batch put / delete requests Aug 25, 2025
@lance6716
fix lint
5600349 
Signed-off-by: lance6716 <lance6716@gmail.com>
@lance6716
Copy link
Contributor Author

I also found that batch delete will also meet "raft entry is too large" error when cleaning up test data. So I implement Batchable for RawBatchDeleteRequest. PTAL again @pingyu @sleepymole

pingyu
pingyu reviewed Aug 25, 2025
View reviewed changes
@lance6716
check batch delete really clean up data
dffa44e 
Signed-off-by: lance6716 <lance6716@gmail.com>
@lance6716
Copy link
Contributor Author

/unhold

@ti-chi-bot ti-chi-bot bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 25, 2025
pingyu
pingyu approved these changes Aug 25, 2025
View reviewed changes
Copy link
Collaborator

@pingyu pingyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM~

@ti-chi-bot ti-chi-bot
Copy link

ti-chi-bot bot commented Aug 25, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: crazycs520, pingyu, sleepymole

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added the lgtm label Aug 25, 2025
@ti-chi-bot ti-chi-bot bot removed the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Aug 25, 2025
@ti-chi-bot ti-chi-bot
Copy link

ti-chi-bot bot commented Aug 25, 2025

[LGTM Timeline notifier]

Timeline:

  • 2025-08-25 09:16:23.411931214 +0000 UTC m=+172475.289762158: ☑️ agreed by crazycs520.
  • 2025-08-25 13:41:39.571573217 +0000 UTC m=+188391.449404170: ☑️ agreed by pingyu.

@ti-chi-bot ti-chi-bot bot merged commit 76959e7 into tikv:master Aug 25, 2025
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pingyu pingyu pingyu approved these changes

+2 more reviewers

@sleepymole sleepymole sleepymole approved these changes

@crazycs520 crazycs520 crazycs520 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved dco-signoff: yes Indicates the PR's author has signed the dco. lgtm size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@lance6716 @pingyu @sleepymole @crazycs520