Memory leak in LRU cache?

[ankgup87]

Hi,

I am seeing a strange memory leak issue with LRUCache. When LRUCache capacity is set as 8MB, then process's memory space remain constant and does not grow with time. When LRUCache's capacity is set > 8MB then process's memory space slowly grows and is ultimately killed by OS.

DB used for testing

DB already has 650M rows where each row's key is around 20 bytes and value averaging around 250 bytes. DB operations are put and multiGet. Keys used for multi-gets are chosen in random fashion. Rocksdb is loaded as part of JAVA process and DB operations are accessed through JNI layer. Cache is not exposed through JNI layer and default cache used by rocksdb is used.

Cache testing

1. Default cache of rocksdb (8MB capacity) is used and process's memory size remains constant.
2. When cache size is bumped up to say 512MB, process's memory space slowly grows and is finally killed by OS. Cache capacity was bumped up by changing db/db_impl.cc:
   result.block_cache = NewLRUCache(67108864);

Please note that between 1) and 2), there were no changes in code other than increasing cache capacity.

Can someone please comment if they have seen this behavior? Thanks.!

PS: I am using latest code of rocksdb from github repo. Other options used for creating DB:

Write buffer size: 128MB
DisableSeekCompaction: false
block size: 4KB
maxWriteBufferNumber: 3
Background compaction threads: 5
level0 file num compaction trigger: 4
level 0 stop files trigger: 12
numLevels: 4
maxGrandparent Overlap Factor: 10
Max bytes for level base: 1GB
No Compression
Custom compaction filter

[dhruba]

This is very interesting.

db/db_impl.cc: result.block_cache = NewLRUCache(67108864);

At first, i thought that you have a sign-extension problem with the parameter that you are passing to NewLRUCache, where you are specifying a v v large size for the cache. But maybe not.

Can you pl send me the first 500 lines of your log file (named LOG)? It will list the configs that the DB is using.

[ankgup87]

Hi Dhruba,

Thanks for following up on this issue. I actually found a memory leak in JNI layer (will update JNI repo with fix).

Will close the issue. Thanks again and apologies for raising a false alarm :)