Initial support for secondary cache in LRUCache #8271

anand1976 · 2021-05-05T20:40:00Z

Defined the abstract interface for a secondary cache in include/rocksdb/secondary_cache.h, and updated LRUCacheOptions to take a std::shared_ptr. An item is initially inserted into the LRU (primary) cache. When it ages out and evicted from memory, its inserted into the secondary cache. On a LRU cache miss and successful lookup in the secondary cache, the item is promoted to the LRU cache. Only support synchronous lookup currently. The secondary cache would be used to implement a persistent (flash cache) or compressed cache.

Tests:
Results from cache_bench and db_bench don't show any regression due to these changes.

cache_bench results before and after this change -
Command
./cache_bench -ops_per_thread=10000000 -threads=1
Before
Complete in 40.688 s; QPS = 245774
Complete in 40.486 s; QPS = 246996
Complete in 42.019 s; QPS = 237989
After
Complete in 40.672 s; QPS = 245869
Complete in 44.622 s; QPS = 224107
Complete in 42.445 s; QPS = 235599

db_bench results before this change, and with this change + #8213 and #8191 -
Commands
./db_bench --benchmarks="fillseq,compact" -num=30000000 -key_size=32 -value_size=256 -use_direct_io_for_flush_and_compaction=true -db=/home/anand76/nvm_cache/db -partition_index_and_filters=true

./db_bench -db=/home/anand76/nvm_cache/db -use_existing_db=true -benchmarks=readrandom -num=30000000 -key_size=32 -value_size=256 -use_direct_reads=true -cache_size=1073741824 -cache_numshardbits=6 -cache_index_and_filter_blocks=true -read_random_exp_range=17 -statistics -partition_index_and_filters=true -threads=16 -duration=300
Before

DB path: [/home/anand76/nvm_cache/db]                                                                                                                                        
readrandom   :      80.702 micros/op 198104 ops/sec;   54.4 MB/s (3708999 of 3708999 found)

DB path: [/home/anand76/nvm_cache/db]                                                                                                                                        
readrandom   :      87.124 micros/op 183625 ops/sec;   50.4 MB/s (3439999 of 3439999 found)

After

DB path: [/home/anand76/nvm_cache/db]                                                                                                                                        
readrandom   :      77.653 micros/op 206025 ops/sec;   56.6 MB/s (3866999 of 3866999 found)

DB path: [/home/anand76/nvm_cache/db]                                                                                                                                        
readrandom   :      84.962 micros/op 188299 ops/sec;   51.7 MB/s (3535999 of 3535999 found)

pdillinger

Although the design is general, it seems to make the "simple" or "normal" case complicated.

Normally, we store a mix of "parsed" (e.g. ParsedFullFilterBlock or Block) and "no parsing necessary" (BlockContents) data structures in block cache.

I expect the "normal" case for tiered cache to be storing just the BlockContents, perhaps with recompression or something. Do we have plans for something more elaborate?

Did we consider an alternate design like this: instead of adding an "unparsing" layer to get block cache data to tiered cache, codify the parsing part of the volatile tier into the block cache itself, so that the tiered cache has direct access to the unparsed version (BlockContents)? It seems like this would involve exposing some CacheableEntry / BlocklikeTraits functionality into block cache (seems reasonable) and save the ugliness (and execution cost?) of coding up (and executing) an unparsing step.

include/rocksdb/cache.h

pdillinger · 2021-05-06T16:16:25Z

cache/lru_cache.cc

+  // mutex if we're going to lookup in the NVM cache
+  // Only support synchronous for now
+  // TODO: Support asynchronous lookup in NVM cache
+  if (!e && tiered_cache_ && helper_cb && wait) {


LRUCache is among the most performance-sensitive pieces of code we have. Have you evaluated the performance impact of these changes (e.g. cache_bench)? Do you see an efficient path to planned changes?

Good point. I'll run benchmarks to check for regression.

include/rocksdb/tiered_cache.h

cache/clock_cache.cc

cache/lru_cache.cc

cache/lru_cache.h

pdillinger · 2021-05-06T16:34:25Z

cache/lru_cache.h

  void Free() {
    assert(refs == 0);
-    if (deleter) {
-      (*deleter)(key(), value);
+    if (!IsTieredCacheCompatible() && info_.deleter) {


It also seems awkward wedging in this new capability into one implementation, rather than extending it to (perhaps) more naturally support the data.

pdillinger · 2021-05-06T17:10:45Z

It seems like this would involve exposing some CacheableEntry / BlocklikeTraits functionality into block cache (seems reasonable) ...

Except perhaps for Cache API compatibility, which I would argue is not so critical.

anand1976 · 2021-05-06T22:04:40Z

Although the design is general, it seems to make the "simple" or "normal" case complicated.

Normally, we store a mix of "parsed" (e.g. ParsedFullFilterBlock or Block) and "no parsing necessary" (BlockContents) data structures in block cache.

We store BlockContents only in the compressed block cache. The uncompressed block cache mostly has objects that require some parsing.

I expect the "normal" case for tiered cache to be storing just the BlockContents, perhaps with recompression or something. Do we have plans for something more elaborate?

For the time being, yes. We may be able to do something more elaborate with PM, like a deep copy, so it'd be good to retain the flexibility.

Did we consider an alternate design like this: instead of adding an "unparsing" layer to get block cache data to tiered cache, codify the parsing part of the volatile tier into the block cache itself, so that the tiered cache has direct access to the unparsed version (BlockContents)? It seems like this would involve exposing some CacheableEntry / BlocklikeTraits functionality into block cache (seems reasonable) and save the ugliness (and execution cost?) of coding up (and executing) an unparsing step.

Interesting. I haven't considered this design. I'm not sure the block cache would need to be exposed to the BlocklikeTraits functionality. The Insert API could just take a pointer to the raw data right, since the assumption is that no special parsing is needed?

I'm not too concerned about the execution cost of "unparsing", since its just extracting the raw data pointer. The bigger overhead is memcpy'ing the data. OTOH, I think its too restrictive to assume that the tiered cache would only ever need access to the BlockContents/raw data.

facebook-github-bot · 2021-05-11T17:21:53Z

@anand1976 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

include/rocksdb/cache.h

cache/lru_cache.cc

zhichao-cao

Thanks for working on it. In general it looks good for me. The title and description need to be updated to "Secondary cache". Since it is experimental, maybe we do not need to log the change to HISTORY.md for now?

facebook-github-bot · 2021-05-12T17:54:21Z

@anand1976 has updated the pull request. You must reimport the pull request before landing.

zhichao-cao

LGTM, thanks!

facebook-github-bot · 2021-05-12T23:30:31Z

@anand1976 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot · 2021-05-13T00:18:04Z

@anand1976 has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2021-05-13T00:28:18Z

@anand1976 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

pdillinger

I didn't dig into tests, but I'm now satisfied with the interface and indications of no notable performance regression.

include/rocksdb/cache.h

pdillinger · 2021-05-13T15:52:34Z

include/rocksdb/cache.h

+  // its not ready. The caller should then call Value() to check if the
+  // item was successfully retrieved. If unsuccessful (perhaps due to an
+  // IO error), Value() will return nullptr.
+  virtual Handle* Lookup(const Slice& key, const CacheItemHelper* /*helper_cb*/,


With the waiting functionality, consider (probably future PR) whether it's worth fixing the race condition between failed lookup and insert. Although our "redundant" block cache insertion statistics suggest it's not a problem currently, with more moves toward remote storage, we could run into more "thundering herd" type problems. My prototype is here: pdillinger@513affa#diff-e9d84127b0d5363ff842cce3fca54dfbffec5d9955f810e7678d69c5c5904e3fR227

include/rocksdb/cache.h

facebook-github-bot · 2021-05-13T21:32:01Z

@anand1976 has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2021-05-13T21:32:21Z

@anand1976 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot · 2021-05-13T22:53:38Z

@anand1976 has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2021-05-13T22:53:56Z

@anand1976 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot · 2021-05-14T01:55:25Z

@anand1976 has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2021-05-14T01:58:11Z

@anand1976 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot · 2021-05-14T05:58:51Z

@anand1976 merged this pull request in feb06e8.

…ache (#8315) Summary: Secondary cache is implemented to achieve the secondary cache tier for block cache. New Insert and Lookup APIs are introduced in #8271 . To support and use the secondary cache in block based table reader, this PR introduces the corresponding callback functions that will be used in secondary cache, and update the Insert and Lookup APIs accordingly. benchmarking: ./db_bench --benchmarks="fillrandom" -num=1000000 -key_size=32 -value_size=256 -use_direct_io_for_flush_and_compaction=true -db=/tmp/rocks_t/db -partition_index_and_filters=true ./db_bench -db=/tmp/rocks_t/db -use_existing_db=true -benchmarks=readrandom -num=1000000 -key_size=32 -value_size=256 -use_direct_reads=true -cache_size=1073741824 -cache_numshardbits=5 -cache_index_and_filter_blocks=true -read_random_exp_range=17 -statistics -partition_index_and_filters=true -stats_dump_period_sec=30 -reads=50000000 master benchmarking results: readrandom : 3.923 micros/op 254881 ops/sec; 33.4 MB/s (23849796 of 50000000 found) rocksdb.db.get.micros P50 : 2.820992 P95 : 5.636716 P99 : 16.450553 P100 : 8396.000000 COUNT : 50000000 SUM : 179947064 Current PR benchmarking results readrandom : 4.083 micros/op 244925 ops/sec; 32.1 MB/s (23849796 of 50000000 found) rocksdb.db.get.micros P50 : 2.967687 P95 : 5.754916 P99 : 15.665912 P100 : 8213.000000 COUNT : 50000000 SUM : 187250053 About 3.8% throughput reduction. P50: 5.2% increasing, P95, 2.09% increasing, P99 4.77% improvement Pull Request resolved: #8315 Test Plan: added the testing case Reviewed By: anand1976 Differential Revision: D28599774 Pulled By: zhichao-cao fbshipit-source-id: 098c4df0d7327d3a546df7604b2f1602f13044ed

anand1976 requested review from siying and zhichao-cao May 5, 2021 20:40

facebook-github-bot added the CLA Signed label May 5, 2021

pdillinger reviewed May 6, 2021

View reviewed changes

anand1976 requested a review from pdillinger May 11, 2021 00:31

zhichao-cao reviewed May 11, 2021

View reviewed changes

include/rocksdb/cache.h Outdated Show resolved Hide resolved

zhichao-cao reviewed May 11, 2021

View reviewed changes

include/rocksdb/cache.h Outdated Show resolved Hide resolved

zhichao-cao reviewed May 11, 2021

View reviewed changes

cache/lru_cache.cc Show resolved Hide resolved

zhichao-cao reviewed May 12, 2021

View reviewed changes

anand1976 force-pushed the tiered_cache branch from 4002fe9 to d9054f9 Compare May 12, 2021 17:54

zhichao-cao approved these changes May 12, 2021

View reviewed changes

anand1976 changed the title ~~Initial support for tiered cache in LRUCache~~ Initial support for secondary cache in LRUCache May 12, 2021

pdillinger approved these changes May 13, 2021

View reviewed changes

anand76 added 8 commits May 13, 2021 15:52

Initial tiered cache support in LRUCache

8b25b69

Commit tiered_cache.h

aaf70ca

Address comments

2be5893

Fix some asan test failures

ed98908

Address more review comments

6117c5f

Fix rebase

9e69439

Change isReady to IsReady to match RocksDB naming convention

6d32ba9

Address some more comments

db0a552

anand1976 force-pushed the tiered_cache branch from 25d5924 to db0a552 Compare May 13, 2021 22:53

Fix some lint warnings

8fd9215

facebook-github-bot closed this in feb06e8 May 14, 2021

facebook-github-bot added the Merged label May 14, 2021

zhichao-cao mentioned this pull request May 18, 2021

Use new Insert and Lookup APIs in table reader to support secondary cache #8315

Closed

anand1976 mentioned this pull request May 28, 2021

Implement a non-volatile SecondaryCache #8347

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initial support for secondary cache in LRUCache #8271

Initial support for secondary cache in LRUCache #8271

anand1976 commented May 5, 2021 •

edited

Loading

pdillinger left a comment

pdillinger May 6, 2021

anand1976 May 6, 2021

pdillinger May 6, 2021

pdillinger commented May 6, 2021

anand1976 commented May 6, 2021

facebook-github-bot commented May 11, 2021

zhichao-cao left a comment

facebook-github-bot commented May 12, 2021

zhichao-cao left a comment

facebook-github-bot commented May 12, 2021

facebook-github-bot commented May 13, 2021

facebook-github-bot commented May 13, 2021

pdillinger left a comment

pdillinger May 13, 2021

facebook-github-bot commented May 13, 2021

facebook-github-bot commented May 13, 2021

facebook-github-bot commented May 13, 2021

facebook-github-bot commented May 13, 2021

facebook-github-bot commented May 14, 2021

facebook-github-bot commented May 14, 2021

facebook-github-bot commented May 14, 2021

Initial support for secondary cache in LRUCache #8271

Initial support for secondary cache in LRUCache #8271

Conversation

anand1976 commented May 5, 2021 • edited Loading

pdillinger left a comment

Choose a reason for hiding this comment

pdillinger May 6, 2021

Choose a reason for hiding this comment

anand1976 May 6, 2021

Choose a reason for hiding this comment

pdillinger May 6, 2021

Choose a reason for hiding this comment

pdillinger commented May 6, 2021

anand1976 commented May 6, 2021

facebook-github-bot commented May 11, 2021

zhichao-cao left a comment

Choose a reason for hiding this comment

facebook-github-bot commented May 12, 2021

zhichao-cao left a comment

Choose a reason for hiding this comment

facebook-github-bot commented May 12, 2021

facebook-github-bot commented May 13, 2021

facebook-github-bot commented May 13, 2021

pdillinger left a comment

Choose a reason for hiding this comment

pdillinger May 13, 2021

Choose a reason for hiding this comment

facebook-github-bot commented May 13, 2021

facebook-github-bot commented May 13, 2021

facebook-github-bot commented May 13, 2021

facebook-github-bot commented May 13, 2021

facebook-github-bot commented May 14, 2021

facebook-github-bot commented May 14, 2021

facebook-github-bot commented May 14, 2021

anand1976 commented May 5, 2021 •

edited

Loading