Skip to content

Commit

Permalink
Reduce contention in parallel eviction by counting opportunistically/…
Browse files Browse the repository at this point in the history
…inaccurately which is acceptable...

Just substract the difference and report if it's positive, as in, the count after eviction is lower, it can be higher due to overlap with a lot of insertions, in which case we can simply report 0.
Ideally, we would use a spaced counter with per-core hashed offsets to prevent false sharing but that would be hard to do reliably due to enumerator.AsParallel() behavior. Don't forget that this is a naive implementation imposed by inherent limitations of ConcurrentDictionary API. It still gives significant (+2x) performance improvement on multi-core systems but is otherwise less efficient due to SPMC behavior of the underlying parallel logic.
  • Loading branch information
neon-sunset committed Jun 26, 2023
1 parent 92bd1e3 commit c436740
Showing 1 changed file with 5 additions and 4 deletions.
9 changes: 5 additions & 4 deletions src/FastCache.Cached/CacheManager.cs
Original file line number Diff line number Diff line change
Expand Up @@ -320,17 +320,15 @@ internal static void ReportEvictions(uint count)
{
var now = TimeUtils.Now;
var store = CacheStaticHolder<K, V>.Store;
uint totalRemoved = 0;
var countBefore = store.Count;

void CheckAndRemove(KeyValuePair<K, CachedInner<V>> kvp)
{
var (key, timestamp) = (kvp.Key, kvp.Value._timestamp);
ref var count = ref totalRemoved;

if (now > timestamp)
{
store.TryRemove(key, out _);
count++;
}
}

Expand All @@ -339,7 +337,10 @@ void CheckAndRemove(KeyValuePair<K, CachedInner<V>> kvp)
.AsUnordered()
.ForAll(CheckAndRemove);

return totalRemoved;
// Perform dirty evictions count and discard the result if eviction
// happened to overlap with significant amount of insertions.
// The logic that consumes this value does not require precise reports.
return (uint)Math.Max(countBefore - store.Count, 0);
}

private static async ValueTask ConsiderFullGC<T>()
Expand Down

0 comments on commit c436740

Please sign in to comment.