Improve Resilience and Exception Support Features

[cruftex]

The current behavior is described here:

• About Caching Exceptions
• http://stackoverflow.com/a/33863701/2594725
• Conditional expiry of entries (e.g.: don't expire if CacheSource is unavailable) #25

In short:

• Resilience: Exceptions originating from the cache source (loader in JSR107 lingo) can be suppressed, if old data is available
• Caching and rethrowing of exceptions: If no old data is available exceptions are cached and rethrown on a get. Other caches without exceptions support would query the data source again and again, which might lead to a worse load in case there is a temporary problem with the data source

Both behaviors are enabled by default.

Problems:

• While the features are useful and make an application more robust, it dangerously may hide permanent problems, e.g. caused by packed loss, below the cache layer. There should be better monitoring / notification options.
• Rethrowing a cached exception causes confusion, since the exception actually happened earlier
• In case of a permanent exception the old value will be returned forever. At some point in time it may be better to discard the old value and propagate the exception, so it becomes obvious that there is a permanent failure.

Enhancement ideas:

ExceptionPropagator

On access an exception is rethrown as PropagatedCacheException. Make this configurable. I think there is limited use to rethrow the original exception, since the stack trace gets crippled, but it might be useful to customize the exception text.

SourceExceptionListener

Gets called for every exception that actually happened. This is useful for adding logging. Lower priority, can be done by wrapping the cachesource, if needed.

CacheConfig.setMaximumStaleTime

Maximum time span the cache is allowed to return stale data.

ResiliencePolicy.propageteException(long lastFetch, V oldData, long now, Throwable exception)

Configurable policy that decides whether to propagate the exception or return stale data. Prio: I would prefer the single stale time parameter rather then making it fully flexible.

CacheInfoMXBean.getStaleEntryCount()

The number of entries with stale data (stale means expired and not able to retrieve fresh one).

CacheInfoMXBean.getExceptionsEntryCount()

The number of entries containing exceptions.

CacheInfoMXBean.getAlert() / CacheManagerInfoMX.getAlert()

Alert is the single consolidated health status that can be monitored. Return warning, if more the 20% of a cache content is stale data or cached exceptions.

Cache.invalidate(K key)

Alternative to Cache.remove() for read through configuration. Invalidates an entry, so it gets retrieved from the source on the next access. However, invalidate does not remove the old data, so in case of an exception the cache can still return it.

[cruftex]

Closing, ResiliencePolicy was added in 0.26. JMX metrics that expose the number of entries with exceptions are missing and will be added later.