Refactor cache package and interfaces to something intentional

[56quarters]

The current cache package is a mix of things that came from multiple packages with varying levels of abstraction in Thanos. We didn't really clean them up or decide what an appropriate cache interface would look like when removing the dependency on Thanos and copying code into dskit. Let's do that now.

I propose, in roughly the following order:

• memcachedClient and redisClient are renamed to MemcachedClient and RedisClient respectively and the constructors for these structs return *MemcachedClient and *RedisClient instead of the interface RemoteCacheClient. This will mean they are exported for consumers of dskit to use.
  • Export Memcached and Redis client types #453
• Remove legacy metrics emitted by Memcached clients
  • Remove deprecated metrics emitted by Memcached client #461
• We standardize on a single "cache" interface and gradually migrate the various implementations to it. Example:

  type Cache interface {
      Get(ctx context.Context, key string, opts ...Option) []byte
      GetMulti(ctx context.Context, keys []string, opts ...Option) map[string][]byte
      SetAsync(key string, value []byte, ttl time.Duration) error
      SetMultiAsync(data map[string][]byte, ttl time.Duration) error
      Delete(ctx context.Context, key string) error
      Stop()
      Name() string
   }

  • This involves collapsing the distinction between MemcachedClient and MemcachedCache and the corresponding Redis implementations. The remoteCache struct isn't doing much except for emitting a few metrics. We'll have to make sure any metrics emitted by these various things are preserved through the migration.
    • Move requests/hits metrics to cache clients #465
    • Remove errors from set methods on RemoteCacheClient interface #466
    • Cache: Make the purpose of remoteCache adapter explicit #471
    • cache: Remove duplicate cache interfaces and update implementations #520