Ability to cache mark and index files for S3 disk #13076
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Added cache layer for DiskS3 (cache to local disk mark and index files).
Detailed description / Documentation draft:
Introduced cache decorator over an IDisk.
Cache decorator tries to cache files by given predicated to a provided local disk.
writeFile() first writes a file to cache and after writing is finished copies this file to underlying disk.
readFile() tries to read from a cache first. If the file doesn't exist it initiates downloading this file from underlying disk to cache and when downloading is finished - returns read buffer from the cache.
Files moving, renaming, deleting are preserved in cache layer as well to prevent cache pollution with stale files.