Improve performance of MemoryCache by using ConcurrentDictionary rather than ReaderWriterLockSlim

[mikeharder]

It may be possible to significantly improve the performance of MemoryCache by using ConcurrentDictionary rather than Dictionary protected by a ReaderWriterLockSlim. Under high load, there is contention acquiring the read lock from ReaderWriterLockSlim.

I created a quick prototype which shows significantly improved performance, though it may not be functionally correct in all scenarios.

Scenario	MemoryCache Implementation	RPS	CPU
Plaintext	N/A	1,167,369	100%
MemoryCachePlaintext	ReaderWriterLockSlim	342,129	60%
MemoryCachePlaintext	ConcurrentDictionary	802,076	100%

The remaining work is to improve and test the ConcurrentDictionary implementation to ensure it's correct under high load, has no race conditions, etc.

Prototype: https://github.com/aspnet/Caching/tree/mikeharder/concurrent-dictionary

[Dissimilis]

Please thoroughly test ConcurrentDictionary vs lock'ed Dictionary performance and don't make this change blindly. Performance may depend a lot on specific workload (amount of get vs set, etc.), even simple Monitor may be faster than ReaderWriterLockSlim if most operations are writes.

[JunTaoLuo]

@Dissimilis thanks for the feedback but is there any particular workload you are concerned about? A few scenarios we have benchmarked include 1) concurrent gets with a few set to update the entry and 2) concurrent sets and removes and have seen major improvements in both. See the referenced PR for benchmark results. The move toward ConcurrentDictionary also takes advantage of the more fine-grained locking and will significantly improve performance when concurrently accessing different entries in the cache compared to the previous implementation which uses a single ReaderWriterLockSlim across the entire dictionary.

[sebastienros]

Can we update the table with the new numbers?

[JunTaoLuo]

Scenario	previous CPU	previous RPS	CPU	RPS
benchmarks: memorycacheplaintext	60	342,129	100	886,544
benchmarks: memorycacheplaintextsetremove	-	-	100	689,634