Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support SST Decoder #284

Merged
merged 2 commits into from
Oct 14, 2021
Merged

Support SST Decoder #284

merged 2 commits into from
Oct 14, 2021

Conversation

marsishandsome
Copy link
Collaborator

@marsishandsome marsishandsome commented Oct 9, 2021 •
edited

Signed-off-by: marsishandsome marsishandsome@gmail.com

What problem does this PR solve?

close #285

support decode SST to Key-Value

subtask of tikv/tikv#11007

What is changed and how it works?

RocksDB provides a java API SstFileReader to decode SST to Key-Value pairs. We can wrapper this API in client-java.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Code changes

  • Has exported function/method change
  • Has exported variable/fields change
  • Has interface methods change
  • Has persistent data change

Side effects

  • Possible performance regression
  • Increased code complexity
  • Breaking backward compatibility

Related changes

  • Need to cherry-pick to the release branch
  • Need to update the documentation
  • Need to be included in the release note

@marsishandsome marsishandsome force-pushed the feature/support-sst-decoder branch 6 times, most recently from 4bcdc15 to 6cb7646 Compare October 11, 2021 02:50
@marsishandsome
Support SST Decoder
e0c00f6 
Signed-off-by: marsishandsome <marsishandsome@gmail.com>
@marsishandsome
Copy link
Collaborator Author

/run-all-tests

@marsishandsome marsishandsome mentioned this pull request Oct 11, 2021
Merged
Little-Wallace
Little-Wallace approved these changes Oct 13, 2021
View reviewed changes
Copy link
Contributor

@Little-Wallace Little-Wallace left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@andylokandy
Copy link
Contributor

andylokandy commented Oct 13, 2021 •
edited

I suggest checking SSTMeta.data_encode. This should help avoid silent bugs after enabling API V2.

@marsishandsome
Copy link
Collaborator Author

marsishandsome commented Oct 14, 2021 •
edited

I suggest checking SSTMeta.data_encode. This should help avoid silent bugs after enabling API V2.

Cool. Can we do this job as a subtask of API V2?
We can update to API to pass the SST Metadata file and leave the implementation empty.

@marsishandsome
Copy link
Collaborator Author

/run-all-tests

@marsishandsome
Copy link
Collaborator Author

/run-all-tests

@marsishandsome
add backupmeta decoder
5869fb8 
Signed-off-by: marsishandsome <marsishandsome@gmail.com>
@marsishandsome
Copy link
Collaborator Author

/run-all-tests

@marsishandsome marsishandsome merged commit bbe4f9b into tikv:master Oct 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andylokandy andylokandy andylokandy approved these changes

@Little-Wallace Little-Wallace Little-Wallace approved these changes

@ti-srebot ti-srebot ti-srebot approved these changes

Assignees
No one assigned
Labels
status/LGT2
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

support decode SST to Key-Value
4 participants
@marsishandsome @andylokandy @Little-Wallace @ti-srebot