Eviction listener

[ksperling]

Similar to #38, it would be really useful to have an eviction listener.

In my use-case i'm using an in-memory Cache<MyKey, File> to manage an on-disk cache of files, so whenever an entry is expired / removed / evicted from the cache I need to delete the corresponding file.

I'm currently using caffeine but would like to try cache2k to see if it supports our access patterns better.

[cruftex]

Thanks for the use case. At the current state an eviction listener isn't to complicated to provide, but I am hesitating, once it is there, I have to support it forever.

I am a little in doubt whether "this is it". Maybe its an XY-Problem?

Thinking deeper about your use case, I get the impression that you actually like to use the eviction algorithm and not the cache. So one question is what data is actually in memory and what features is the "whole cache" adding? Maybe a reusable library for the eviction is a better alternative?

If its an interactive application, there is also the restart problem. With a restart the whole history is lost. I don't know how critical this is for your application. Supporting the use case the next logical request would be to somehow preserve/persist the cache state over a restart. What happens to the disk files on shutdown?

Thoughts?

[ksperling]

Yeah I have thought of that, however it is not just the eviction that I am using, it's also the read-through capability that guarantees that multiple concurrent requests for the same entry result only in a single attempt to load it. If there was an "eviction" library that supported all this, I imagine it would still have to manage a certain amount of per-item meta-data storage internally, in addition to whatever storage of the actual content I am doing myself, so I'm not sure it would look that different in practice from just using a cache to store a 'handle' to the actual data.

As far as restarting the application goes, the files remain on disk and get re-added to the cache on startup, so any frequency / recency meta-data is lost. In practice that doesn't seem to be a huge issue so far. In our use case data actually has an intrinsic timestamp and recent data is much more likely to be accessed than older data, so if it was possible to expose the intrinsic age of each item to the cache in some way (rather than the cache using time since load/write to define recency) this would be very powerful for our use case and make the loss of access frequency meta-data even less of an issue.

In terms of eviction, another use case might be having a cache of Bitmap objects on Android (or anything else that references non-JVM memory) that require a recycle() or similar deallocation call. Intuitively that feels like less of an 'abuse' of an in-memory cache, but conceptually there's really very little difference between a JNI pointer to native memory and a file name / handle. For any of these cases it would be nice to have a generic removal listener that can be notified for replacement / removal / expiry / eviction in a uniform way.

[cruftex]

Thanks for the detailed thoughts on this. Looks reasonable.

so if it was possible to expose the intrinsic age of each item to the cache

cache2k honors the least recently used aspect. So when you insert the oldest ones first, that will do.

I plan to release 1.0 now with the current feature set. Then I like to start an 1.1 dev version. This allows testing new features, while being able to change and improve the API or even drop features again that didn't prove. I cannot promise, but probably something testable will pop up the next months. Would be nice when you then take a look at it and test it.

[srikrishnacj]

I too need this feature badly. My scenario is use cache like swap memory for OS.

[cruftex]

@srikrishnacj
Can you describe more about your use case? What should happen when something is read or written to the cache? What should happen on eviction?

[srikrishnacj]

I cannot explain my business usecase but I will try to explain as close as possible. I Have to huge number of records from file. Each record creates a new object or updates the existing objects one or more created by previous records. My Computer memory is so small to process all the objects in memory.If use a disk based storage like mongo or couch, its takes very huge amount of time says in hours, since huge amount of lookup and update calls. So I want to use cache primary storage and hard disk as swap space and do all the operation on cache. All the creation and updates are done on the cache. If cache is full, I will save the evicted or expired items to disk. If cache does not have object, then I will load from by disk via CacheLoader. Once the process is completed, I will iterate all the objects in cache and flush it to disk. I have my own implementation for this scenario with HashMap. But it is not advanced like it does not track least used elements or timeout expire. Once the memory is full, it flushes everything to disk and caches the data one by one from disk if required via get(). another draw back is getting memory size via system call is painful. So I am trying to my custom cache with Standard JCache Implementation. So basically I want CacheLoader, EvictionListener, ExpireListener and eager expire and eviction policy. While searching for standard libs i came up with cache2k and caffeine. caffeine seems to fulfill all my needs. caffeine seems to full fill all my needs and as bones it seamlessly evicts least used objects during garbage collection with soft values configuration enabled . cache2k is more interesting since it supports JCache out of the box and performance bonues. only lag is eviction listener.

[ivanviragine]

My use case is: I have a lot of drivers that takes a long time to load and takes up a lot of memory. So, everytime I need a new driver, I get it from cache or load it and put in the cache.
As this drivers takes a lot of memory, I keep it in the cache only for some minutes, to save loading time if I may need it again. When the time comes, it should be evicted, but, for a proper eviction, I should run the close method on the cached driver, so it shuts down gracefully.
I've ended up putting a very large capacity, to avoid eviction and put a cache expiry listener as this:

.addListener(new CacheEntryExpiredListener<String, CachedDriver>() {
@OverRide
public void onEntryExpired(final Cache<String, CachedDriver> cache, final CacheEntry<String, CachedDriver> entry) {
entry.getValue().close();
}
})

This works as expected, only drawback is that I can't use a small or specific capacity.

[cruftex]

Thanks for the more detailed descriptions!

@srikrishnacj
You describe something like storage tiers. Using an eviction listener can be tricky, since there might be race conditions. For your use case to work it is important that the eviction listener is synchronous and blocks if a new request wants to load the entry again.

I also can think alternative support for your use case. E.g. a write behind like feature, that uses the cache writer for writing changes but only when entries are evicted or there is spare storage bandwidth. However, this will take more time to implement.

@ivanviragine
That is the "free additional resources" use case, same as @ksperling described. Yes, seams useful.

[srikrishnacj]

@cruftex Sorry for long delayed response.

there is race condition but i will guard it with read write re entrant lock. Exactly as you said i will write entries on evicted. like cache as a swap storage. That's why i want eviction listener on cache.