Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

storage: make ingest and split_region interface compatible with API v2 #12389

Merged
merged 129 commits into from May 16, 2022
Merged

storage: make ingest and split_region interface compatible with API v2 #12389

merged 129 commits into from May 16, 2022

Conversation

iosmanthus
Copy link
Member

@iosmanthus iosmanthus commented Apr 20, 2022 •
edited

What is changed and how does it work?

  1. Add a ts field for RawWriteBatch if the API version is v2.
  2. Encode the split key if the API version is v2.

Issue Number: ref #10938

What's Changed:

make ingest and split_region interface compatible with API v2.

Related changes

  • Need to cherry-pick the release branch

Check List

Tests

  • Unit test
  • Integration test

Side effects

  • Performance regression
    • Consumes more CPU
    • Consumes more MEM

Release note 

storage: make ingest and split_region interface compatible with API v2

pingyu and others added 30 commits February 24, 2022 21:37
@pingyu
wip
94b5a14 
Signed-off-by: pingyu <yuping@pingcap.com>
@pingyu
cdc: implement causal timestamp provider
3c324ed 
Issue Number: ref tikv#11965

Signed-off-by: pingyu <yuping@pingcap.com>
@pingyu
wip
80c9b2c 
Signed-off-by: pingyu <yuping@pingcap.com>
@pingyu
wip
0c8319f 
Signed-off-by: pingyu <yuping@pingcap.com>
@pingyu
api_version: add codec for raw key.
6a40169 
Issue Number: ref tikv#11965

Signed-off-by: pingyu <yuping@pingcap.com>
@pingyu
Merge branch 'master' into feat-rawkv-cdc-api-ver
8aed19f
@pingyu
Merge remote-tracking branch 'upstream/master' into feat-rawkv-cdc-ca…
b4109cc 
…usal-ts

Signed-off-by: pingyu <yuping@pingcap.com>
@pingyu
Merge branch 'feat-rawkv-cdc-api-ver' into feat-rawkv-cdc-causal-ts
de61907 
Signed-off-by: pingyu <yuping@pingcap.com>
@pingyu
api_version: improve prefix verification
de1d226 
Issue Number: ref tikv#12036

Signed-off-by: pingyu <yuping@pingcap.com>
@pingyu
Merge branch 'master' into feat-rawkv-cdc-api-ver
c1818b3
@pingyu
Merge branch 'feat-rawkv-cdc-api-ver' into feat-rawkv-cdc-causal-ts
e152861 
Signed-off-by: pingyu <yuping@pingcap.com>
@pingyu
api_version: refactor to using txn_types::Key
8613440 
Issue Number: ref tikv#11965

Signed-off-by: pingyu <yuping@pingcap.com>
@pingyu
api_version: address comments.
a02891a 
Issue Number: ref tikv#11965

Signed-off-by: pingyu <yuping@pingcap.com>
@andylokandy
Merge branch 'master' into feat-rawkv-cdc-api-ver
5b7d954
@pingyu
Merge remote-tracking branch 'origin/feat-rawkv-cdc-api-ver' into fea…
0c68160 
…t-rawkv-cdc-causal-ts

Signed-off-by: pingyu <yuping@pingcap.com>
@pingyu
wip
93165c5 
Signed-off-by: pingyu <yuping@pingcap.com>
@pingyu
Merge branch 'master' into feat-rawkv-cdc-causal-ts
ba5fbae 
Signed-off-by: pingyu <yuping@pingcap.com>
@pingyu
Merge branch 'master' into feat-rawkv-cdc-api-ver
648b64e
@pingyu
Merge branch 'feat-rawkv-cdc-api-ver' into feat-rawkv-cdc-causal-ts
c663c22 
Signed-off-by: pingyu <yuping@pingcap.com>
@pingyu
causal_ts: Add causal timestamp component for RawKV CDC
d8d6a42 
Issue Number: ref tikv#11965

Signed-off-by: pingyu <yuping@pingcap.com>
@pingyu
Merge remote-tracking branch 'upstream/master' into feat-rawkv-cdc-ca…
e7e6434 
…usal-ts

Signed-off-by: pingyu <yuping@pingcap.com>
@pingyu
causal_ts: bug fix.
6de9adb 
Issue Number: ref tikv#11965

Signed-off-by: pingyu <yuping@pingcap.com>
@pingyu
wip
f71e1ce 
Signed-off-by: pingyu <yuping@pingcap.com>
@pingyu
wip
d2bf71d 
Signed-off-by: pingyu <yuping@pingcap.com>
@pingyu
rawkv: Add pre-propose callback to append causal timestamp. Enable ca…
fa511ed 
…usal timestamp management.

Issue Number: ref tikv#11965

Signed-off-by: pingyu <yuping@pingcap.com>
@pingyu
Merge remote-tracking branch 'upstream/master' into feat-rawkv-cdc-ra…
77aba80 
…w-get-put

Signed-off-by: pingyu <yuping@pingcap.com>
@pingyu
wip
c66fc0f 
Signed-off-by: pingyu <yuping@pingcap.com>
@pingyu
wip
68b4268 
Signed-off-by: pingyu <yuping@pingcap.com>
@haojinming @pingyu
api_v2: raw_delete
2065e93 
Signed-off-by: pingyu <yuping@pingcap.com>
@pingyu
Merge remote-tracking branch 'upstream/master' into feat-rawkv-cdc-ra…
b057a26 
…w-get-put

Signed-off-by: pingyu <yuping@pingcap.com>
@ti-chi-bot
Copy link
Member

@iosmanthus: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@iosmanthus
Copy link
Member Author

/run-all-tests

@iosmanthus
Copy link
Member Author

/run-all-tests

@pingyu
Copy link
Contributor

pingyu commented May 13, 2022

/run-test

@iosmanthus
Copy link
Member Author

/merge

@ti-chi-bot
Copy link
Member

@iosmanthus: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@pingyu
Copy link
Contributor

pingyu commented May 13, 2022

/run-test

1 similar comment
@pingyu
Copy link
Contributor

pingyu commented May 13, 2022

/run-test

@pingyu
Copy link
Contributor

pingyu commented May 13, 2022

/run-all-tests

@ti-chi-bot
Copy link
Member

@iosmanthus: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@pingyu
Copy link
Contributor

pingyu commented May 14, 2022

/run-test

2 similar comments
@pingyu
Copy link
Contributor

pingyu commented May 14, 2022

/run-test

@pingyu
Copy link
Contributor

pingyu commented May 14, 2022

/run-test

@pingyu
Copy link
Contributor

pingyu commented May 15, 2022

/run-all-tests

@ti-chi-bot
Copy link
Member

@haojinming: Thanks for your review. The bot only counts approvals from reviewers and higher roles in list, but you're still welcome to leave your comments.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot ti-chi-bot merged commit ec904c1 into tikv:master May 16, 2022
@pingyu pingyu mentioned this pull request May 16, 2022
Closed
51 tasks
fengou1 pushed a commit to fengou1/tikv that referenced this pull request May 26, 2022
@iosmanthus @pingyu
@andylokandy @haojinming @ti-chi-bot @fengou1
storage: make ingest and split_region interface compatible with API v2 (
9cb393d 
tikv#12389)

ref tikv#10938

make ingest and split_region interface compatible with API v2.

Signed-off-by: pingyu <yuping@pingcap.com>
Signed-off-by: haojinming <jinming.hao@pingcap.com>
Signed-off-by: Andy Lok <andylokandy@hotmail.com>
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>

Co-authored-by: pingyu <yuping@pingcap.com>
Co-authored-by: Ping Yu <yuping@pingcap.com>
Co-authored-by: Andy Lok <andylokandy@hotmail.com>
Co-authored-by: haojinming <jinming.hao@pingcap.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pingyu pingyu pingyu approved these changes

@overvenus overvenus overvenus approved these changes

@3pointer 3pointer 3pointer approved these changes

@haojinming haojinming haojinming approved these changes

Assignees
No one assigned
Labels
release-note size/XL status/can-merge Status: Can merge to base branch status/LGT2 Status: PR - There are already 2 approvals
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
7 participants
@iosmanthus @ti-chi-bot @pingyu @overvenus @3pointer @haojinming @andylokandy