Cache eviction for CHMCache, or another default cache

[joewreschnig]

Right now CHMCache has no eviction policy - when it fills up, it never changes. But our GeoIP lookups are not just clustered (so benefiting from a cache) but also time-varying (benefiting from the cache changing over time). E.g. from around UTC 0:00 to 6:00, we switch from mostly-European to mostly-North-American IPs.

I understand that CHMCache is supposed to be mostly an example of how to write a cache, but we're dealing with applications written in JRuby, and that makes writing our own cache a little more painful and a lot slower.

Would it be possible to offer a default cache with an eviction policy?

[oschwald]

As you mention, the CHMCache is primarily an example of how to implement a cache, but it does work ok for most basic used cases as things like countries, continents, and major cities end up in the cache. We would be open to alternate cache implementations or enhancements to CHMCache as long as they didn't pull in new dependencies (e.g., no Apache Commons).

[cameronkerrnz]

I'm not a Java programmer, but this looks promising, and standard to Java 7 at least: http://chriswu.me/blog/a-lru-cache-in-10-lines-of-java/ implements a LRU Cache based on a LinkedHashMap in a mode (selected by constructor) to be ordered by access time.

https://docs.oracle.com/javase/7/docs/api/java/util/LinkedHashMap.html#LinkedHashMap(int,%20float,%20boolean)

Worth noting from the docs:

Note that this implementation is not synchronized. If multiple threads access a linked hash map concurrently, and at least one of the threads modifies the map structurally, it must be synchronized externally. This is typically accomplished by synchronizing on some object that naturally encapsulates the map. If no such object exists, the map should be "wrapped" using the Collections.synchronizedMap method. This is best done at creation time, to prevent accidental unsynchronized access to the map:

Map m = Collections.synchronizedMap(new LinkedHashMap(...));

A structural modification is any operation that adds or deletes one or more mappings or, in the case of access-ordered linked hash maps, affects iteration order. In insertion-ordered linked hash maps, merely changing the value associated with a key that is already contained in the map is not a structural modification. In access-ordered linked hash maps, merely querying the map with get is a structural modification.)

[re-thc]

Why not use something like https://github.com/ben-manes/caffeine ?

[oschwald]

We do not want to pull in a bunch of dependencies that many users will never use. Caffeine, in particular, also requires Java 8, while we still support 7. If you would like to use Caffeine, it is very easy to implement your own cache using CHMCache as an example.

[ben-manes]

Maybe consider ConcurrentLinkedHashMap if you want a tiny dependency, or an older (JDK7) Guava if already used?

[oschwald]

We tried implementing an LRU cache and the performance was strictly worse when looking up random IPs than CHMCache, at least for the default cache size and larger sizes. The reason is that even with random lookups, CHMCache ends up caching most of the relevant data, e.g., the country maps, and it has an extremely low overhead. Managing eviction, especially in a thread-safe manner, ends up being almost as costly as just decoding the data from the database.

Given this, I am going to close this issue without prejudice . Future PRs with alternate caching implementations will be considered on their merits.