-
Notifications
You must be signed in to change notification settings - Fork 6.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Eliminate some redundant block reads. #3354
Eliminate some redundant block reads. #3354
Conversation
Re-use metadata for reading Compression Dictionary on BlockBased table open, this saves two reads from disk.
@yuslepukhin has updated the pull request. View: changes |
Just curious, are you using dictionary compression in production? If so, are you able to share any details about the benefit you see on compression ratio? We don't use it yet but hope to some day. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ajkr has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
@ajkr I take that back. We are not using compression dictionary and the perf gain comes from issuing parallel reads in async version. This PR may not be that useful after all. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It eliminates one of my TODOs, so let's take it anyways :). LGTM, thanks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ajkr is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
Summary: Re-use metadata for reading Compression Dictionary on BlockBased table open, this saves two reads from disk. This helps to our 999 percentile in 5.6.1 where prefetch buffer is not present. Closes facebook#3354 Differential Revision: D6695753 Pulled By: ajkr fbshipit-source-id: bb8acd9e9e66e65b89c548ab8940570ae360333c
Re-use metadata for reading Compression Dictionary on BlockBased
table open, this saves two reads from disk.
This helps to our 999 percentile in 5.6.1 where prefetch buffer is not present.