Shared block cache in RocksJava

[sagar0]

Changes to support sharing block cache using the Java API.

Previously DB instances could share the block cache only when the same Options instance is passed to all the DB instances. But now, with this change, it is possible to explicitly create a cache and pass it to multiple options instances, to share the block cache.

Implementing this for Rocksandra, but this feature has been requested by many java api users over the years.

Test Plan:

• Added a unit test to make sure shared block cache is working.
• Added another simple test to make sure that setBlockCache() is working.

Sample code to show how it works:

import java.lang.IllegalArgumentException;

import org.rocksdb.*;
import org.rocksdb.util.SizeUnit;

public class SharedCacheSample
{
  static {
    RocksDB.loadLibrary();
  }

  public static void main( String[] args ) throws Exception {
    if (args.length < 2) {
      System.out.println("usage: SharedCacheSample db_path num_shards");
      System.exit(-1);
    }

    String baseDBPath = args[0];
    int numShards = 0;
    try {
      numShards = Integer.parseInt(args[1]);
    } catch (NumberFormatException e) {
      System.out.println("Enter a good value for num_shards");
      throw e;
    }

    try {
      Statistics stats = new Statistics();
      Cache cache = new LRUCache(64 * SizeUnit.MB);
      for (int shard = 0; shard < numShards; shard++) {
        BlockBasedTableConfig table_options = new BlockBasedTableConfig();
        table_options.setBlockCache(cache);
        Options options = new Options();
        options.setCreateIfMissing(true)
               .setStatistics(stats)
               .setTableFormatConfig(table_options);

        String db_path = baseDBPath + "/" + shard;
        RocksDB db = null;
        db = RocksDB.open(options, db_path);
        db.put("hello".getBytes(), "world".getBytes());

        FlushOptions flushOptions = new FlushOptions();
        db.flush(flushOptions);

        byte[] value = db.get("hello".getBytes());
        assert ("world".equals(new String(value)));

        System.out.println(stats.getTickerCount(TickerType.BLOCK_CACHE_ADD));
      }
    } catch (IllegalArgumentException e) {
      assert (false);
    } catch (RocksDBException e) {
      System.out.format("[ERROR] caught an unexpected RocksDB exception -- %s\n", e);
      throw e;
      // assert (false);
    } catch (Exception e) {
      assert (false);
    }
  }
}

Build and Run:

javac -g -verbose -cp ~/rocksdb/java/target/rocksdbjni-5.12.0-linux64.jar SharedCacheSample.java
java -cp ~/rocksdb/java/target/rocksdbjni-5.12.0-linux64.jar:. SharedCacheSample /tmp/rocksjavabc 8
1
2
3
4
5
6
7
8  

In the above output, you can see the counter for the block cache is increasing even though they are different DB instances.

Note that all the block caches have the same memory address below:

svemuri@devbig187 ~ $ ls -l /tmp/rocksjavabc
total 32
drwxr-xr-x. 2 svemuri users 4096 Mar 16 09:54 0
drwxr-xr-x. 2 svemuri users 4096 Mar 16 09:54 1
drwxr-xr-x. 2 svemuri users 4096 Mar 16 09:54 2
drwxr-xr-x. 2 svemuri users 4096 Mar 16 09:54 3
drwxr-xr-x. 2 svemuri users 4096 Mar 16 09:54 4
drwxr-xr-x. 2 svemuri users 4096 Mar 16 09:54 5
drwxr-xr-x. 2 svemuri users 4096 Mar 16 09:54 6
drwxr-xr-x. 2 svemuri users 4096 Mar 16 09:54 7

svemuri@devbig187 ~ $ cat /tmp/rocksjavabc/*/LOG | egrep "\sblock_cache:"
  block_cache: 0x7f214c50a2a8
  block_cache: 0x7f214c50a2a8
  block_cache: 0x7f214c50a2a8
  block_cache: 0x7f214c50a2a8
  block_cache: 0x7f214c50a2a8
  block_cache: 0x7f214c50a2a8
  block_cache: 0x7f214c50a2a8
  block_cache: 0x7f214c50a2a8

svemuri@devbig187 ~ $ cat /tmp/rocksjavabc/*/LOG | egrep -A5 "\sblock_cache:"
  block_cache: 0x7f214c50a2a8
  block_cache_name: LRUCache
  block_cache_options:
    capacity : 67108864
    num_shard_bits : 6
    strict_capacity_limit : 0
--
  block_cache: 0x7f214c50a2a8
  block_cache_name: LRUCache
  block_cache_options:
    capacity : 67108864
    num_shard_bits : 6
    strict_capacity_limit : 0
--
  block_cache: 0x7f214c50a2a8
  block_cache_name: LRUCache
  block_cache_options:
    capacity : 67108864
    num_shard_bits : 6
    strict_capacity_limit : 0
--
  block_cache: 0x7f214c50a2a8
  block_cache_name: LRUCache
  block_cache_options:
    capacity : 67108864
    num_shard_bits : 6
    strict_capacity_limit : 0
--
  block_cache: 0x7f214c50a2a8
  block_cache_name: LRUCache
  block_cache_options:
    capacity : 67108864
    num_shard_bits : 6
    strict_capacity_limit : 0
--
  block_cache: 0x7f214c50a2a8
  block_cache_name: LRUCache
  block_cache_options:
    capacity : 67108864
    num_shard_bits : 6
    strict_capacity_limit : 0
--
  block_cache: 0x7f214c50a2a8
  block_cache_name: LRUCache
  block_cache_options:
    capacity : 67108864
    num_shard_bits : 6
    strict_capacity_limit : 0
--
  block_cache: 0x7f214c50a2a8
  block_cache_name: LRUCache
  block_cache_options:
    capacity : 67108864
    num_shard_bits : 6
    strict_capacity_limit : 0

[facebook-github-bot]

@sagar0 has updated the pull request. View: changes

[facebook-github-bot]

@sagar0 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

[benclay]

Nice. We've been using the same/copied Options trick, this will be cleaner.

[facebook-github-bot]

@sagar0 has updated the pull request. View: changes, changes since last import

[facebook-github-bot]

@sagar0 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

[wpc]

Read throught , PR looks solid good. Merge it? Can't wait test it in our cluster :-)

[facebook-github-bot]

@sagar0 is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.