Implement invalidation component

[verdie-g]

Problem

• Since TryPeek can return expired values and do the reloading in background, one can get a value that was expired for ages from a key that is unfrequently accessed. The effect is worse when many cache instances are involved.
• TryGetAsync can reload the key from the source in foreground which will impact the 99 percentile of an application.

Invalidation use-cases

External service sending invalidation notification

An external service is aware of changes in the data source and send invalidation messages. For instance:

1. The datasource is a PostgreSQL table and for each insert/update/delete, a trigger sends a notification containing the primary key of the changed row. The notification should be listened by all cache instances and reload the key accordingly.
2. Cache instances elect a leader that will be in charge of polling a PostgreSQL table for row version changes and send notifications to other instances.

Invalidation notification on distributed cache backfilling

By using the redis notifications, the first cache instance that will reload a key and backfill the distributed cache (redis here) will trigger a redis notification that would be listened by all instances to refresh their local cache.

This is not perfect but it reduce initial problems impact.

Manually invalidate a known invalid key

For a cache of IPEndPoint, if the returned endpoint is not accessible it may be because the entry is invalid and a refresh could help.

Solutions

Alternative #1

interface IInvalidator<TKey>
{
    IAsyncEnumerable<TKey> GetInvalidatedKeys(CancellationToken);
}

This component should get invalidations from a user-defined stream (Redis notification, PostgreSQL notification, Kafka message, RabbitMQ message) of keys and invalidate the local entry associated with the key.

When TryPeeking an invalidated entry, it should return it to the user but the background reload should skip the distributed cache. When TryGettingAsync the invalidated entry it should directly call the data source loader.

Problems

All cache instances will hit the data source even when a distributed cache is available. Usually the number of invalidated keys is low so it shouldn't be a real problem.

Alternative #2

record Invalidation<TKey>(TKey Key, bool FreshInDistributedCache);

interface IInvalidator<TKey>
{
    IAsyncEnumerable<Invalidation<TKey>> GetInvalidatedKeys(CancellationToken);
}

Same as the first but the user can specify if the distributed cache contains a fresh value so rather than all cache instances reload the key from the data source, they can reload it from the distributed cache.

Problems

The user need a way to set the key in the distributed cache. IDistributedCache interface should probably be exposed in some way.