How to get the list of keys using IDistributedCache extension.

[RajivPandi]

Is there an extension method available to get the list of keys from the cache.
I am using Redis Cache provider, i can able to get the list of keys matching the patter using Redis CLI. I am looking for similar feature using IDistributedCache.

[chuanboz]

+1 for this request. Any plan to add this support in the near future?

[stefankip]

Too bad nobody cares to help is with this 👎

[skironDotNet]

You could write your own. I know not a good answer but I'm looking for same thing, so I'm going to have my instance of IDistributedCache where I pass original IDistributedCache and low level Redis client, then will have a method to get keys, keeping ready features of IDistributedCache like sliding expiration management.

Saying all that I'm supprised MS, didn't make this by deafult like KeyExists() or didn't expose Redis client as a property of IDistributedCache so it would be super easy to write extensions to use every feature of Redis.

[dropsonic]

It is a very important feature indeed.
In my scenario, I have some features in every cache item (e.g., locale, username, etc.), and I want to invalidate the cache by a specific feature (e.g., invalidate all cache items where locale = "en-US").
It is impossible to do with the current interface.

@skironDotNet exposing Redis client is not a good idea because IDistributedCache hides implementation details from you. It might be Redis, but it also might be SQL or in-memory cache so you cannot use Redis-specific details.

[maryamariyan]

I believe this would be a new feature request.

cc: @Tratcher @eerhardt

[Tratcher]

Yes this would require new APIs. We've decided against this in the past for both IDistributedCache and IMemoryCache because of the ephemeral nature of cache entries. An entry name might be returned in the list, but then be removed and/or replaced before you could request it.

[eerhardt]

We've decided against this in the past for both IDistributedCache and IMemoryCache because of the ephemeral nature of cache entries.

My thinking is that we could return a "snapshot" of the keys in the cache at the point in time you ask for them. There wouldn't be a guarantee that the key would still exist (or have the same value) after the call. The idea being you could use this new API in conjunction with #36568, to allow users to remove many entries in a single call. Given the snapshot, they can look through them (maybe apply Regex or StartsWith or similar), and build up a list of the entries they want to remove. This seems to be a pretty popular scenario - we saw a handful of issues discussing this same thing.

[rickdgray]

How is this still not implemented? Disappointed.

[maryamariyan]

Set milestone to 8.0. Next step here we'll need to prepare API suggestions.

[Strandedpirate]

When you don't dogfood your own code, and basic requirements fall through the cracks for literal years... oof
See you all in .net 10 when this finally drops, and we've all retired.

[dario-oppl]

this would require new APIs. We've decided against this in the past for both IDistributedCache and IMemoryCache because of the ephemeral nature of cache entries. An entry name might be returned in the list, but then be removed and/or replaced before you c

But let me decide about what do I do in that case!
Can't believe this is THE reason for not adding the method.
I can't imagine rdbms not implementing SELECT because they implement a DELETE :( and data can be deleted before is queried.

[BuddhaBuddy1]

Just give us a Command or Action method. That way, it can be atomic, not a returned list of values, but an action that is performed during the scan at the moment a matching key is found.

[WhitWaldo]

I would urge the team to think less about downstream ramifications of returning keys that may no longer be present and leave that as an exercise to the developer about how to handle the issue. Presumably, we're aware (since we're querying a cache implementation) that we're requesting information about keys whose values are bound to various expiration windows.

I would thus propose that the revised API add both a 1) method for listing known, current (at that time, knowing it's a distributed operation and this snapshot may not be perfectly up to date) keys stored in the system making no guarantees about whether they'll be available in a future call or not and 2) a method (e.g. GetCacheItemMetadata or the like) that allows the developer to, given a key, get information about the expiration window for a given cache item, if it is still valid when this call is made.

If the expiration information is meaningful to the developer, this gives them an opportunity to do that second round-trip to the implementation to both 1) determine if the entry is still valid and 2) learn how much longer it will be valid for.

Otherwise, when requesting a list of keys with no guarantees made about their lifetime after that ask, any calls to get the value would succeed or fail just like any other time I attempt to access a cache entry with a valid or expired key: you don't know until you ask.

I don't see any reason to overcomplicate the ask here, but I'd be interested to hear more about why the team thinks such guardrails are necessary here.

[wilkovanderveen]

So if I understand correctly, the only way to retrieve multiple cached items by an expression based on the key, is, for now, to know and use the concrete implementation behind the interface for allowing to get all keys which match a certain pattern?

[davidhenley]

Is this still not done after 5 years? I guess back to using the IConnectionMultiplexer itself

var db = Redis.GetDatabase();
var endpoint = Redis.GetEndPoints().First();
var redisServer = Redis.GetServer(endpoint);
await foreach (var key in redisServer.KeysAsync(pattern: $"{typeof(TEntity).Name}:list*"))
{
    await db.KeyDeleteAsync(key);
}

[acandaldev]

With a concurrentDictionary and adding deleting in Get and Remove can be done?
I did my personal implementation of this

[mgravell]

net9-timescale update; we're discussing a range of IDistributedCache changes here; to be explicit, "list of keys" support is not currently on the "doing right now" list, but: the idea that we might want to support this is duly noted. The fundamental idea on that ticket is that IDistributedCache is not a particularly useful API, and the plan is to add a higher-level HybridCache which uses IDistributedCache as a backend. With that perspective, it means that we could also add some optional ISomeNewKeysWhatever API that the backend can implement if appropriate, and a suitable "get keys if you can" method on HybridCache. And since that is a base class rather than an interface, we can add methods without it breaking anything.

However: I'll be honest - I'm not a huge fan of "list the keys" APIs; in reality, most times I've seen people use this kind of API: they've been using it inappropriately and in ways that are actively harmful to the system. Key-value stores usually aren't optimized for this scenario if it is even possible at all - they're optimized for "GET {key}" and "PUT {KEY} {value} {expiry}" (in the language of the backend). This usage might make sense for a "database admin / visualization" tool, but it isn't usually something you should use in routine/regular application code, IMO.

I will note that Aspire makes it easier to also get to the "real" backend, such as IConnectionMultiplexer in the case of redis, and from there: there are ways of getting the keys - although it would be up to the consumer to enforce DB/prefix/etc complications, which are the things we would deal with on the caller's behalf if we did add direct support. Obviously this doesn't generalize - i.e. if you're using SQL or Cosmos or NCache or whatever: you'd need to start from scratch. The "add some ISomeNewKeysWhatever that HybridCache knows about" would generalize, but would still be dependent on individual backends adding support (we would help with that for some backends).