Multiple cachelib instances or multiple pools

[wenhaocs]

Hi, I have a question on creating cache using CacheLib.
In our scenario, we want to set up multiple caches for different objects in a single daemon. I read from the CacheLib website:

If you're using multiple cache instances, please consider if you can simplify to using a single instance via multiple cache pools. CacheLib is highly optimized for concurrency; using multiple cachelib instances is typically an anti-pattern unless you have a good reason.

If I understand correctly, if I set them as pools of the same cache instance, I can achieve a better performance than setting them as individual cache instance. Am I correct? But in this case, these pools have to use the same eviction policy, as they are added to the same cache trait (e.g., LruCacheTrait, Lru2QCacheTrait, etc.). What if I want them to have configurable eviction policies? Please advise. Thanks!

[ss7pro]

I'm not part of Meta cachelib team but we did some benchmarking and noticed that as you add more threads to instance performance per thread goes down. This is mostly related to locks that needs to be held during evictions when you add a new item and there's not enough memory to accomodate that. So if you plan to put significant amount of load on your cache and run many threads you're better separating data types to different instances.

[haowu14]

Hi @wenhaocs, your understanding is correct that different pools will be using the same cache trait thus the same eviction policy.

On multiple instances vs. multiple pools: The benefit of multiple pools is that cachelib can help you balance the memory between pools. If you use multiple instances, you will need to make sure you size each instances and adjust them should your workload changes.

[wenhaocs]

Make sense. Thanks @haowu14 @ss7pro