Support using secondary cache with the blob cache #10349
Conversation
gangliao
requested review from
ltamasi,
akankshamahajan15,
riversand963 and
gitbw95
|
Blob cache should use the same secondary cache as used in LRU cache backed or they should be 2 separate Cachelib instances? |
Hi @zhichao-cao, we haven't tested it yet. But ideally, both modes can be supported. |
|
@gangliao has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@gangliao has updated the pull request. You must reimport the pull request before landing. |
|
@gangliao has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Summary: RocksDB supports a two-level cache hierarchy (see https://rocksdb.org/blog/2021/05/27/rocksdb-secondary-cache.html), where items evicted from the primary cache can be spilled over to the secondary cache, or items from the secondary cache can be promoted to the primary one. We have a CacheLib-based non-volatile secondary cache implementation that can be used to improve read latencies and reduce the amount of network bandwidth when using distributed file systems. In addition, we have recently implemented a compressed secondary cache that can be used as a replacement for the OS page cache when e.g. direct I/O is used. The goals of this task are to add support for using a secondary cache with the blob cache and to measure the potential performance gains using `db_bench`.
|
@gangliao has updated the pull request. You must reimport the pull request before landing. |
|
@gangliao has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Summary:
RocksDB supports a two-level cache hierarchy (see https://rocksdb.org/blog/2021/05/27/rocksdb-secondary-cache.html), where items evicted from the primary cache can be spilled over to the secondary cache, or items from the secondary cache can be promoted to the primary one. We have a CacheLib-based non-volatile secondary cache implementation that can be used to improve read latencies and reduce the amount of network bandwidth when using distributed file systems. In addition, we have recently implemented a compressed secondary cache that can be used as a replacement for the OS page cache when e.g. direct I/O is used. The goals of this task are to add support for using a secondary cache with the blob cache and to measure the potential performance gains using
db_bench.This task is a part of #10156