About the api design

[He-Pin]

Hi, super thank for this great library, I have to question about the API design.

1. Why there is a put but not a remove on the Cache? and I always want to use a val cachedValue = cache.remove(key) like thing.
2. In the performance wiki, I saw the performance is better than the CCHM for compute, get, just for performance and ignoring the memory overhead, seems like I can always use caffeine's Cache as CCHM.

Thank you, and have a nice day.

[ben-manes]

Why there is a put but not a remove on the Cache? and I always want to use a val cachedValue = cache.remove(key) like thing.

A cache removal is usually called an invalidation so we have Cache.invalidate(key). This is a void method because best practice is to not rely or assume anything about the current contents of the cache and treat it as a transparent, transient storage layer. That makes the Cache interface opinionated. However, options or best practices shouldn't stop a developer from solving their problems, so we have the asMap() view for the familiar Collections interfaces. A Map is typically a passive data structure, e.g. it isn't changing underneath you, whereas a Cache is an active one so we wanted to keep it separate.

In the performance wiki, I saw the performance is better than the CCHM for compute, get, just for performance and ignoring the memory overhead, seems like I can always use caffeine's Cache as CCHM.

Yes, an unbounded cache is a thin wrapper to conform to our interfaces. The benchmark's difference is that in Java 8's ConcurrentHashMap it pessimistically locks on a computeIfAbsent whereas we'll do a fast Map.get to avoid blocking if present (since we've optimized heavily for good hit rates). You can always write those few lines yourself, but otherwise it is nice to not worry when the cache dependency is already present. Mostly it is good to know that computeIfAbsent could be a bottleneck in high throughput situations.

You can read Doug's reasoning, where he relented for Java 9 by adding a partial pre-screening if the desired entry is at the head of the hashbin. That somewhat reduces the chance of blocking but doesn't resolve the performance issue. I believe with the removal of biased locking and reduction of safepoints in favor of thread-local handshakes that his concerns no longer true. You could email him if interested in having him revisit that design choice for future JDKs. Last I talked with him he was thinking about redesigning for per-entry locking for computes instead of the hashbin, so it's reasonable timing.

[He-Pin]

Thanks for the details, I am asking this because we are currently building a lock step gaming server, and some parts are using the CCHM and some are using Caffeine Cache, and there is a module which is using 'ConcurrentHashMap<String,ConcurrentHashMap<String, Subscription>>', and i think with Caffeine Cache would be better, which can makes life easier and provides memory safety, so i was planning to touch that part of code.

And yes, without an abstraction of 'AsyncCache', which do cause some problems, when there is a heavy nearly 50ms computation for loading data from Redis with HSCAN.

Thank you very much.

[ben-manes]

Sorry, I forgot to mention of Q1,

Ideally we want to promote loading through the cache and not using Cache.put. That avoids cache stampedes where multiple threads see an entry is missing, load it, and put on top of each other. While the result is likely idempotent, it could cause a query storm on the system of record (e.g. database). That is a very common reason for an outage at scale, e.g. Facebook and others have hit it multiple times. A put is perfectly valid so we provide it, but we don't want to emphasize users performing explicit put and remove operations when computeIfAbsent and an eviction policy can solve their problems implicitly.