Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Proposal]Optimize OpenAndTrimHistory performance with skip unnessary compacting files by checking each file's timestamp meta #9863

Open
sunlike-Lipo opened this issue Apr 19, 2022 · 0 comments
Labels

Comments

@sunlike-Lipo
Copy link
Contributor

sunlike-Lipo commented Apr 19, 2022

A new API named DB::OpenAndTrimHistory is added to imported by (#9223). This API will open DB and trim data to the timestamp specified by trim_ts (The data with newer timestamp than specified trim bound will be removed). Currently this API will iterate every sst file to trim data which has newer timestamp than specified trim_ts. Actually files with max_ts smaller than trim_ts should be safe and no need to be iterated.

Since we can track sst file's max and min timestamp according (#8959),
I think the OpenAndTrimHistory API can be optimized by skipping unnessary sst files with call API CompactFiles instead of CompactRangeInternal.

@riversand963

@ajkr ajkr added question up-for-grabs Up for grabs labels Apr 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants