memory for cache while using maximumSize

[Snelly2458]

Hi ,
I am currently using caffeine for in memory caching in Java.
i wanted to know how the memory is allocated for the cache?
i understand that maximumSize , is for number of entries which works to evict entry’s , but in this case how can I know the memory that would be assigned for the Cache?

would adding the weigher and providing a memory value , make sure that the allotted memory is within the maximumWeight?

[ben-manes]

The cache uses heap memory by layering on top of ConcurrentHashMap. It wraps the value to capture additional per-entry metadata, like the expiration timestamp, and holds a few per cache instance data structures (like the frequency histogram). The metadata overhead is fairly inexpensive and a rough evaluation is captured on the wiki.

As you said, a weigher lets you calculate the cost of the key/value pair. That's actually quite tricky on the JVM (see JEP-8249196) so we don't provide an explicit strategy. You can use java object layout, jamm, etc for a rough estimate.

Typically to understand the tradeoff of how large the cache should be requires analyzing the hit rate curve to decide when you reach a sweet spot. This is usually only if the cache is critical, else a rough guess is made and exported to stats in case something later goes awry. To analyze you can simply log the hash of keys when accessing and run that through the simulator for different cache sizes. Unfortunately most attempts that I have seen to automatically tune cache sizes fail, so layered caches (e.g. caffeine + memcached) is usually a nice pairing to find a good balance. This is especially true because caching can break the generational hypothesis so large caches might incur higher GC overhead, so some will use off-heap values with Caffeine's on heap for policy management. It all depends on the application, infrastructure, ect that makes the most sense as this library is not opinionated in what you consider the best way to use it.

[Snelly2458]

So in case I provide a weigher with a value something like weigher ((k,v)-> 5) , would that mean caffeine would limit it to 5bytes per entry? Or would I have to provide a buffer for the additional metadata?

[ben-manes]

Yes, it means that every entry would take 5 units of capacity out of the total provided. The maximumSize setting simply uses a weigher where every entry takes 1 unit, so per-entry sizing. The accounting does not include anything wrt the cache's own overhead, which is fairly minimal but was an fyi for your awareness.