How to Synchronize Between Cache Put and Removal Listener

[rickilambert]

Hello again! Apologies but i return with more questions!

I was wondering, I have a Cache that has individual expiry for keys as defined below

    private final Cache<String, SomeValue> cache = Caffeine.newBuilder()
            .expireAfter(Expiry.writing((String key, SomeValue) -> Duration.ofDays(1)))
            .scheduler(Scheduler.systemScheduler())
            .removalListener(this::processBadData)
            .build();

The class has a method that essentially calls

cache.policy().expireVariably().orElseThrow().put(key, new SomeValue()), ttl, TimeUnit.SECONDS);

Therefore, every time we get a new message, enter it in to the cache, replace the last entry and reset the time for that key.

This is hit extremely often - within the region of 50 calls a second sometimes more.

There is a slight race condition that's happening (not sure if that's the right terminology here) where by in my logs I am seeing an insert microseconds before an entry is expired and i guess my question is - is it possible to somehow detect that and stop the processBadData removalListener from firing? I guess id need to rearchitect a little? If so how would you suggest please

My second question is - there is quite a variance when looking at the log time stamps - suggesting sometimes the eviction is the time to live up to a second more before the eviction occurs - is there any way to make this more constrained as this needs to be as efficient as possible

Thank you for your time and I hope the above was not too much for a ramble.

[ben-manes]

The cache won't evict an entry if expiration is reset during its removal as that race causes its eviction to be aborted. If you are adding right after it expires then you might consider your own lifecycle variable to detect and no-op in the listener. One approach could be to delay processing the removal notifications until a grace period passes so you can check and skip if it should actually do the work, which you could implement in your custom listener.

Expiration is specified to have a built-in ~1 second (2^30 ns) delay. Internally the cache uses an O(1) priority queue based on hashing, where that is the smallest bucket size, so the entry would expire within that time interval while the bucket expires after the interval ends. The scheduler paces itself to not fire more rapidly and works on the bucket instead of the entries. Java's ScheduledExecutorService uses a min-heap so we get O(lg n) scheduling of the bucket in a shallow tree and O(1) entry expirations. While the last bucket could be instead use a min-heap to be more exact using a similar assumption on costs, the cache does attempt to be a real-time timing subsystem. There will be a variety of delays (such as GC pauses), so either end users need to implement something more exact themselves or build in some tolerances.

Hope that helps!