"LoadingCache.refresh(K)" on a value that hasn't reached the "refresh time" yet?

[raindel]

Hi,

I'm trying to understand some details of .refreshAfterWrite(...) and LoadingCache.refresh(K).

Say, I set up a cache as:

LoadingCache<Key, Graph> productNames = Caffeine.newBuilder()
    .expireAfterWrite(Duration.ofMinutes(10))
    .refreshAfterWrite(Duration.ofMinutes(1))
    .build(key -> getProductName(key));

If I'm reading the documentation right, it goes as this:

• first query for 'key1':
  • product name 'A' loaded into cache
• 1 minute later it becomes eligible for refresh
• 5 minutes pass, in the mean time product name is changed to 'B' in the backend system
• second query for 'key1:
  • Product name 'A' returned to the caller,
  • refresh initiated in the background
  • when refresh succeeds, 'B' is loaded into cache

That is, refresh only affects future calls, but not the one triggered it. is that right?

Then, there's graphs.refresh(key)
If I call it for an entry which is not "refresh eligible" yet (written only 30 seconds ago in the example above), would it still be refreshed? Or that Future would complete immediately, returning "still fresh" value?

[ben-manes]

LoadingCache.refresh(key) is a manual refresh regardless of if refreshAfterWrite. However if one is already in-flight for that key then the call to refresh(key) will no-op.

I think the use-case was if you have a local + remote cache and broadcast an invalidation message on a distributed topic. Then you might proactively reload the contents, e.g. if a configuration change. An Cache.invalidate(key) would cause a new load that blocks calls, which might be undesired latency, while the refresh is an async swap. That's something I used to do with Ehcache + memcached + JMS pre-Guava, may have explained that when we designed Guava's Cache, and it carried over here as a nice API. But in this case I inherited it from past API sessions rather than thought threw it anew. I think the manual refresh is still valid and probably the API would have stayed the same. Sorry that I'm forgetful on this topic.

[ben-manes]

Sorry, busy day and I focused on the last question only.

Yes, the refresh assists for the future requests and not the one that triggered it. Technically if the asyncReload returns an immediate CompletableFuture then its value is returned, but since reloads are non-blocking a concurrent read would still observe the current (stale) value. Since the future may take a while to complete, the current value is returned until the future is resolved or the entry is otherwise evicted (e.g. it expired). In the case where the entry was updated or removed then the reload is abandoned and its value forwarded to the removal listener.

The feature is sometimes referred to as soft vs hard time-to-live (our terms being refresh vs expiration). An expired entry would force the caller to load it anew and all concurrent requests to likewise block. This can be problematic for hot content used on every request, like configuration, which could result in periodic latency spikes. The refresh hides this latency where the current value is still acceptably stale and an async reload can prefetch so that the cached copy stays up to date (an eventual consistency model).

If you broadcast updates as changes occur then instead of waiting for the pull-based reload, you can manually invoke the reload directly. That is still asynchronous, e.g. a Cache.invalidate(key) would block calls needing it, so it gives the ability to push notifications by the key and reload proactively without impacting your latency.

The last model is when you strictly need a periodic full snapshot of the underlying data to be in-memory then that model is a warm replica (hot=change-data-capture, warm=periodic reload, cold=one-time backup). This case is a scheduled task replacing a volatile immutable map, which core Java already provides (ScheduledExecutorService). A lot of users get confused by imagining this model is what we do, but no library is needed and its a replica, not a cache, so the apis/behavior don't align.

[raindel]

Thanks Ben!