New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fs cache improvement for big reads #55158
Conversation
This is an automated comment for commit d384762 with description of existing statuses. It's updated for the latest CI running ❌ Click here to open a full report in a separate page Successful checks
|
b4282ae
to
c7a3c74
Compare
aa1a2e8
to
3c6d829
Compare
3c6d829
to
0907209
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The goal of this PR is to limit the output of getOrSet
method, e.g. we request a certain range and get a holder of file segments from it. But if the range is big, we could request really a lot of file segments. Here we can have two problems as a result:
-
We put an unlimited number of file segments into
FileSegmentsHolder
, e.g. make them unreleasable, but we increase priority of a file segment in the priority queue only right before we process it, and given that file segments fromFileSegmentsHolder
are processed one by one, there can be a lot of file segments which are unreleasable but still on the lowest part of the priority queue, e.g. when cache is full and we want to release something, we will encounter those unreleasable file segments at the lowest priority part of the queue. This slows down eviction especially because of the fact that the number of such file segment is unlimited. -
It often happens that we request a big range but do not read it fully (because of seeks, LIMIT in query, etc).
getOrSet
returns a continuous range of file segments without holes, e.g. we create empty file segments on the place of uncached holes and stores them as potential future downloaded file segment in cache. So if we do not download them after that - they must be removed from cache. But here in case of very big queries we might end up with a huge number of unused zero size file segments which need to be removed. I have seen a case where there were more than 10 thousand such zero size unused and removed from cache file segments, in a single query. What makes it a bit worse - when we erase a file segment, even though its size is known to be 0, we nevertheless check if the corresponding file on the filesystem does not exist. Which makes this removal not as cheap as it could seem.
test_storage_rabbitmq/test.py::test_row_based_formats
01600_parts_states_metrics_long |
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
An improvement which takes place when cache is full and there are big reads. Also improves memory usage of the cache in case of big reads.