Developers can observe notable performance improvements in MemoryCache

[jeffhandley]

With a collection of changes that were identified through profiling, we can gain as much as +15% RPS for the TechEmpower Cached Queries benchmark.

Product improvements

• few minor MemoryCache perf improvements #44797 (low hanging fruits)
• More MemoryCache perf improvements #45280 (more improvements)
• optimize DateTimeOffset.UtcNow by removing redundant verification #45281 (DateTimeOffset.UtcNow +10%)
• Reduce CacheEntry size #45410 (reduce CacheEntry size)
• Remove some allocations related to storing CacheEntry scopes #45563 (reduce allocations)
• Reduce CacheEntry size #45962 (reduce CacheEntry size even further)

Benchmark improvements

• [Caching] remove one additional interface call and a branch aspnet/Benchmarks#1603
• [Caching] remove implicit cast from CachedWord to Word (1.7%) aspnet/Benchmarks#1607
• add MemoryCache.TryGetValue benchmarks performance#1598
• add more MemoryCache benchmarks performance#1606
• add CreateEntry and AddThenRemove_ExpirationTokens benchmarks performance#1618

Proof of concept

• [Proposal] use Timer in MemoryCache #45842 (use Timer)

Bugs Introduced

• Fix PostEvictionCallbacks bug  #46807

Microbenchmark results

Source

Method	Runtime	Mean	Ratio	Allocated
GetHit	.NET 5.0	167.73 ns	1.00	-
GetHit	.NET 6.0	112.15 ns	0.67	-
TryGetValueHit	.NET 5.0	163.23 ns	1.00	-
TryGetValueHit	.NET 6.0	103.77 ns	0.64	-
GetMiss	.NET 5.0	115.63 ns	1.00	-
GetMiss	.NET 6.0	90.21 ns	0.78	-
TryGetValueMiss	.NET 5.0	116.12 ns	1.00	-
TryGetValueMiss	.NET 6.0	92.80 ns	0.80	-
SetOverride	.NET 5.0	394.62 ns	1.00	424 B
SetOverride	.NET 6.0	286.34 ns	0.73	224 B
CreateEntry	.NET 5.0	390.19 ns	1.00	424 B
CreateEntry	.NET 6.0	76.73 ns	0.20	224 B
AddThenRemove_NoExpiration	.NET 5.0	61,943.36 ns	1.00	60,546 B
AddThenRemove_NoExpiration	.NET 6.0	53,270.07 ns	0.86	54,674 B
AddThenRemove_AbsoluteExpiration	.NET 5.0	70,461.50 ns	1.00	73,331 B
AddThenRemove_AbsoluteExpiration	.NET 6.0	55,862.63 ns	0.79	54,290 B
AddThenRemove_RelativeExpiration	.NET 5.0	66,384.71 ns	1.00	61,026 B
AddThenRemove_RelativeExpiration	.NET 6.0	49,905.29 ns	0.75	33,889 B
AddThenRemove_SlidingExpiration	.NET 5.0	70,597.44 ns	1.00	74,099 B
AddThenRemove_SlidingExpiration	.NET 6.0	49,301.41 ns	0.70	40,017 B
AddThenRemove_ExpirationTokens	.NET 5.0	72,186.78 ns	1.00	71,771 B
AddThenRemove_ExpirationTokens	.NET 6.0	65,962.42 ns	0.91	66,490 B

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[ghost]

Tagging subscribers to this area: @eerhardt, @maryamariyan
See info in area-owners.md if you want to be subscribed.

Issue Details

---

With a collection of changes that were identified through profiling, we can gain as much as +15% RPS for the TechEmpower Cached Queries benchmark.

Work Items

• few minor MemoryCache perf improvements #44797 (low hanging fruits)
• More MemoryCache perf improvements #45280 (more improvements)
• optimize DateTimeOffset.UtcNow by removing redundant verification #45281 (DateTimeOffset.UtcNow +10%)
• Reduce CacheEntry size #45410 (reduce CacheEntry size)
• Remove some allocations related to storing CacheEntry scopes #45563 (reduce allocations)
• Reduce CacheEntry size #45962 (reduce CacheEntry size even further)
• Reduce CacheEntry size #45962 (use Timer, most probably will get rejected)

Bugs Introduced

• Fix PostEvictionCallbacks bug  #46807

Author:	jeffhandley
Assignees:	jeffhandley, adamsitnik
Labels:	Bottom Up Work, User Story, area-Extensions-Caching
Milestone:	6.0.0

[ygra]

#45962 (use Timer, most probably will get rejected)

This seems to reference the wrong pull request; it's the same one as the one before, and doesn't seem to have anything to do with a timer. Perhaps #45842 was intended to be linked here?

[jeffhandley]

Thanks, @ygra!

[adamsitnik]

@sebastienros is there any chance that you could add the Caching benchmark to the first page of the PowerBI report?

[adamsitnik]

@jeffhandley I've updated the issue description and closed the proof of concept PR, so our work here is done. I am closing the issue.