Avoiding cache repopulation request storms

Avoiding cache repopulation request storms

ℹ️ This documentation describes the previous Polly v7 API. If you are using the new v8 API, please refer to pollydocs.org.

In high-throughput systems using caching, a concern can arise around request storms to repopulate expensive-to-create cached data.

In a high-frequency system with 100s or 1000s of requests per second, when a popular cache key expires, it is possible that multiple external requests may arrive near-simultaneously which would invoke the underlying call to repopulate that cache key.

If the underlying call to create that data is expensive (ie time-consuming to return the data), multiple parallel requests can potentially mount up, placing additional unnecessary stress on the downstream system.

The Polly-Contrib offers a DuplicateRequestCollapser policy as a way to avoid these cache repopulation storms: RequestCollapserPolicy solves this problem by collapsing concurrent duplicate executions into a single downstream call.

For large server farms, requests can further be collapsed using redis with a distributed lock. Please see Polly-Contrib/Polly.Contrib.DuplicateRequestCollapser.RedisDistributedLock for more information.