NVM cache size usage for fixed size block cache

[pbhandar2]

I am using the following CacheBench configuration I wrote for my custom CacheStressor.

{
"cache_config": {
"cacheSizeMB": 500,
"allocSizes": [4136],
"navySizeClasses": [],
"nvmCachePaths": [
"/users/pbhandar/flash/disk.file"
],
"navyBigHashSizePct": 0,
"navyBlockSize": 4096,
"truncateItemToOriginalAllocSizeInNvm": true,
"nvmCacheSizeMB": 400
},
"test_config": {
"populateItem": true,
"generator": "multi-replay",
"numThreads": 1,
"inputQueueSize": 128,
"processorThreadCount": 8,
"asyncIOTrackerThreadCount": 8,
"traceBlockSizeBytes": 512,
"pageSizeBytes": 4096,
"statPrintDelaySec": 30,
"relativeTiming": true,
"scaleIAT": 100.0,
"diskFilePath": "/users/pbhandar/disk/disk.file",
"replayGeneratorConfig": {
"traceList": ["/users/pbhandar/w103.csv"]
}
}
}

I am replaying a block trace. The size of a page is set to 4096. I had a few questions about configuration and its effect on the system.

The size of each cache request is 4096 bytes. I had to set the allosSizes class for RAM cache to 4136 before the items were being admitted so it seems it took 40 bytes for metadata instead of 31? Also for NVM cache, should I use stack allocation mode and allocate an empty array for class sizes or is there a way to optimize the NVM cache size utilization?

There were around 3.4 million block requests.

blockReqCount=3442095

T1 and T2 were full and had evictions.

t1EvictionCount=4082937
t2EvictionCount=3661448

The size of t1 was 125735 -> 125735 * 4136 bytes -> 520 MB (so slightly more than even allocated?)

t1Size=74161

The size of t2 was 74161->74161 * 4136 bytes -> 306.7 MB (much less than allocated?)

Is there any way I can improve NVM cache size utilization? I know it goes up and down during the experiment but generally I see it around 70%, for even larger NVM sizes like 1.5GB.

[haowu14]

• On RAM allocation size: The required size of an item is the size from allocate call plus the size of the item's key (and also overhead).
  We have deprecated navy size classes and will be removing this config soon. You can leave it an empty array. (And we are always on stack allocation).

Nuance on Navy alignment: navyBlockSize is the alignment when a region gets written to the device. Each region has a memory buffer. By default it is 16MB. The size one item takes in block cache would be 4096+key size + 24 (size of BlockCache::EntryDesc) then aligned to 512. So likely the item ends up taking 4608 bytes. One region would hold 3640 items, meaning that the internal fragmentation in a region is 4096. 4KB in 16MB isn't too bad.
However, when reading one item, you'd need to read two 4096 segments from the device to cover 4608. If you are concerned about the read performance, you should consider shrinking your block size down so that after overheads it is below 4096. Or you can increase the block size so that this overhead isn't too bad compared to the size.

Can you share more how T1 and T2 are setup? Such as how the requests are routed and how do you read the item count and eviction counts.

[haowu14]

You can remove cache_->recordAccess because a) when lookup hits and insert happens, the item will be promoted to the head; b) cache_->recordAccess doesn't promote item to the head, it is for NvmAdmissionPolicy to track access.

On T1 cache, 125735 -> 125735 * 4136 bytes = 495 MB. I think you used 1000 instead of 1024 for the conversion to arrive at 520.

Are T1 and T2 cache both setup by the same config you shared above? How are T1 and T2 cache related, such as is an item missed in T1 will be fetched from T2 and items evicted from T1 will be inserted into T2?

[pbhandar2]

So you are saying that the item is moved to head on the find call if it exists cache_->find(*key, AccessMode::kRead)? I confused recordAccess from some class in CacheLib with recordAccess from CacheBench. So in the following snippet from the CacheStressor in CacheBench, the item is being promoted to head on a get request if it exists and if not is being inserted, right?

    case OpType::kGet: {
      ++stats.get;

      auto slock = chainedItemAcquireSharedLock(*key);
      auto xlock = decltype(chainedItemAcquireUniqueLock(*key)){};

      if (ticker_) {
        ticker_->updateTimeStamp(req.timestamp);
      }
      // TODO currently pure lookaside, we should
      // add a distribution over sequences of requests/access patterns
      // e.g. get-no-set and set-no-get
      cache_->recordAccess(*key);
      auto it = cache_->find(*key, AccessMode::kRead);
      if (it == nullptr) {
        ++stats.getMiss;
        result = OpResultType::kGetMiss;

        if (config_.enableLookaside) {
          // allocate and insert on miss
          // upgrade access privledges, (lock_upgrade is not
          // appropriate here)
          slock = {};
          xlock = chainedItemAcquireUniqueLock(*key);
          setKey(pid, stats, key, *(req.sizeBegin), req.ttlSecs,
                 req.admFeatureMap);
        }
      } else {
        result = OpResultType::kGetHit;
      }

      break;
    }

Here is a dump of the NVM cache settings in more detail. I am storing fixed sized pages so no need for small object cache. I used default settings for DRAM cache.

I0830 12:04:35.884053 150868 Cache-inl.h:207] Using the following nvm config{
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::admissionPolicy": "",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::admissionProbBaseSize": "0",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::admissionProbFactorLowerBound": "0",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::admissionProbFactorUpperBound": "0",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::admissionProbability": "0",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::admissionSuffixLen": "0",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::admissionWriteRate": "0",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::bigHashBucketBfSize": "8",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::bigHashBucketSize": "4096",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::bigHashSizePct": "0",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::bigHashSmallItemMaxSize": "0",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::blockCacheCleanRegions": "1",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::blockCacheDataChecksum": "true",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::blockCacheLru": "true",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::blockCacheNumInMemBuffers": "2",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::blockCacheRegionSize": "16777216",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::blockCacheReinsertionHitsThreshold": "0",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::blockCacheReinsertionPctThreshold": "0",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::blockCacheSegmentedFifoSegmentRatio": "",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::blockSize": "4096",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::deviceMaxWriteSize": "1048576",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::deviceMetadataSize": "0",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::fileName": "/users/pbhandar/nvm/disk.file",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::fileSize": "12645826560",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::maxConcurrentInserts": "1000000",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::maxParcelMemoryMB": "1024",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::maxWriteRate": "0",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::navyReqOrderingShards": "21",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::raidPaths": "",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::readerThreads": "32",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::truncateFile": "false",
I0830 12:04:35.884053 150868 Cache-inl.h:207] "navyConfig::writerThreads": "32"
I0830 12:04:35.884053 150868 Cache-inl.h:207] }
I0830 12:04:36.038064 150868 Factory.cpp:284] Cache file: /users/pbhandar/nvm/disk.file size: 12645826560 truncate: 0
I0830 12:04:36.240216 150868 NavySetup.cpp:191] metadataSize: 63225856. No bighash.
I0830 12:04:36.240341 150868 NavySetup.cpp:121] blockcache: starting offset: 63225856, block cache size: 12566134784
I0830 12:04:36.240660 150868 LruPolicy.cpp:37] LRU policy: expected 749 regions
I0830 12:04:36.249629 150868 RegionManager.cpp:50] 749 regions, 16777216 bytes each
I0830 12:04:36.250527 150868 Allocator.cpp:40] Enable priority-based allocation for Allocator. Number of priorities: 1
I0830 12:04:36.250632 150868 BlockCache.cpp:143] Block cache created
I0830 12:04:36.250702 150868 Driver.cpp:61] Small item cache is noop
I0830 12:04:36.250742 150868 Driver.cpp:64] Max concurrent inserts: 1000000
I0830 12:04:36.250773 150868 Driver.cpp:65] Max parcel memory: 1073741824

Really appreciate the help by the way. Thank you!

[haowu14]

The snippet you posted will promote the item if found and insert the item if missed if enableLookaside is true. Insert usually means insert to the head but could have other variations. For example, see the "insertion point" in LRU: https://cachelib.org/docs/Cache_Library_User_Guides/eviction_policy#lru.

The setup of your NVM cache doesn't seem to have problem. You have 749 regions, which holds a total of 749*16MB data. Do you mind re-iterating the question you are trying to ask here? You mentioned that the utilization of NVM is low. Can you share how you get that observation?

[pbhandar2]

My observation was that the item count in T2 was 74161 for an NVM cache of size 400MB. An NVM cache of 400MB should have 400/16=25 regions. Each region can hold 3640 items if each item is sized 4608; hence the t2 size should be around 3640 * 25 = 91000 but it is around 74161 which is around 20 regions. I have noticed that the size of T2 can change a little even after the cache is full but the size at the end of the experiment is 20 regions, 20% lower than expected which is 25 regions.

What role does size of file in NVM play in NVM cache performance? If I am testing a wide range of sizes, should I have file size adjusted based on the size of NVM cache or should I just use one large file that fits all cases?

[haowu14]

If you look at the logs printed when cachebench started for T2, what is the number X under "LRU Policy: expected X regions"?
My guess is due to metadata size, you don't have full 400MB for block cache and the actual number of region is 20.

You shouldn't need to create a file for cachelib. You can setup nvm cache with navyConfig.setSimpleFile(fileName, fileSize, true) so that cachelib will create a file of fileSize on startup.