Re-evaluating ValueTask and async-only operation

[Turnerj]

Currently with multiple layers, the logic requires frustrating looping and if-checking for the right type of cache layer method to call. We would be able to mostly remove instructions for handling both sync-and-async without gaining additional allocations.

There may be a small regression for in-memory caching however other cache layers may slightly improve in allocations.

Before Benchmarks

CacheStack Benchmark

Method	WorkIterations	Mean [ns]	Error [ns]	StdDev [ns]	Gen 0	Gen 1	Gen 2	Allocated [B]
SetupAndTeardown	1	313.2 ns	6.00 ns	6.42 ns	0.2904	-	-	912 B
Set	1	671.5 ns	10.74 ns	10.04 ns	0.3157	-	-	992 B
Set_TwoLayers	1	903.5 ns	15.07 ns	13.36 ns	0.5608	-	-	1760 B
Evict	1	754.6 ns	10.38 ns	8.67 ns	0.3157	-	-	992 B
Evict_TwoLayers	1	1,066.6 ns	18.13 ns	15.14 ns	0.5608	-	-	1760 B
Cleanup	1	1,130.0 ns	15.63 ns	14.62 ns	0.3567	-	-	1120 B
Cleanup_TwoLayers	1	1,472.6 ns	22.04 ns	20.62 ns	0.6008	-	-	1888 B
GetMiss	1	394.0 ns	6.98 ns	6.53 ns	0.2904	-	-	912 B
GetHit	1	745.1 ns	9.19 ns	8.59 ns	0.3157	-	-	992 B
GetOrSet_NeverStale	1	1,272.9 ns	20.19 ns	17.90 ns	0.4196	-	-	1320 B
GetOrSet_AlwaysStale	1	1,282.1 ns	12.01 ns	10.65 ns	0.4196	-	-	1320 B
GetOrSet_TwoSimultaneous	1	32,048,672.6 ns	366,587.14 ns	306,116.87 ns	-	-	-	2448 B
GetOrSet_FourSimultaneous	1	32,448,376.2 ns	639,241.65 ns	627,820.78 ns	-	-	-	2610 B
SetupAndTeardown	100	345.1 ns	11.62 ns	23.48 ns	0.2904	-	-	912 B
Set	100	34,067.8 ns	655.45 ns	828.93 ns	1.2817	-	-	4160 B
Set_TwoLayers	100	40,025.3 ns	943.27 ns	1,883.81 ns	1.5259	-	-	4928 B
Evict	100	45,558.0 ns	1,421.49 ns	2,805.89 ns	2.8076	-	-	8912 B
Evict_TwoLayers	100	57,106.9 ns	1,304.48 ns	2,385.32 ns	4.5776	-	-	14432 B
Cleanup	100	54,180.1 ns	1,028.07 ns	961.66 ns	8.5449	-	-	26808 B
Cleanup_TwoLayers	100	96,244.3 ns	1,847.18 ns	1,814.17 ns	5.6152	-	-	17728 B
GetMiss	100	6,573.5 ns	99.06 ns	92.66 ns	0.2899	-	-	912 B
GetHit	100	7,453.0 ns	87.04 ns	77.16 ns	0.3128	-	-	992 B
GetOrSet_NeverStale	100	25,816.0 ns	506.45 ns	473.74 ns	0.3967	-	-	1320 B
GetOrSet_AlwaysStale	100	96,324.5 ns	1,877.02 ns	1,927.56 ns	7.4463	-	-	23496 B
GetOrSet_TwoSimultaneous	100	32,647,390.2 ns	642,369.00 ns	713,991.44 ns	-	-	-	17081 B
GetOrSet_FourSimultaneous	100	32,677,946.7 ns	652,984.81 ns	801,923.99 ns	-	-	-	28307 B

MemoryCacheLayer Benchmark

Method	WorkIterations	Mean [ns]	Error [ns]	StdDev [ns]	Gen 0	Gen 1	Gen 2	Allocated [B]
Overhead	1	196.8 ns	3.90 ns	3.83 ns	0.2270	-	-	712 B
GetMiss	1	237.0 ns	4.48 ns	4.19 ns	0.2270	-	-	712 B
GetHit	1	488.5 ns	8.62 ns	8.06 ns	0.2518	-	-	792 B
SetExisting	1	696.2 ns	13.49 ns	12.62 ns	0.2623	-	-	824 B
EvictMiss	1	253.3 ns	5.04 ns	5.17 ns	0.2270	-	-	712 B
EvictHit	1	515.0 ns	9.77 ns	11.63 ns	0.2518	-	-	792 B
Cleanup	1	926.2 ns	18.33 ns	18.01 ns	0.2928	-	-	920 B
Overhead	100	194.9 ns	3.33 ns	3.27 ns	0.2270	-	-	712 B
GetMiss	100	2,722.7 ns	40.64 ns	38.02 ns	0.2251	-	-	712 B
GetHit	100	3,400.4 ns	56.32 ns	52.69 ns	0.2518	-	-	792 B
SetExisting	100	23,858.8 ns	323.26 ns	302.38 ns	1.2512	-	-	3992 B
EvictMiss	100	4,821.1 ns	83.95 ns	74.42 ns	0.2213	-	-	712 B
EvictHit	100	30,345.6 ns	490.47 ns	458.79 ns	2.7466	-	-	8712 B
Cleanup	100	49,275.6 ns	968.67 ns	994.76 ns	12.0850	-	-	37992 B

[Turnerj]

After Benchmarks

CacheStack Benchmark

Method	WorkIterations	Mean [ns]	Error [ns]	StdDev [ns]	Gen 0	Gen 1	Gen 2	Allocated [B]
SetupAndTeardown	1	313.8 ns	5.39 ns	5.05 ns	0.2904	-	-	912 B
Set	1	665.6 ns	8.11 ns	7.59 ns	0.3157	-	-	992 B
Set_TwoLayers	1	892.0 ns	16.26 ns	15.21 ns	0.5608	-	-	1760 B
Evict	1	771.6 ns	14.30 ns	13.37 ns	0.3157	-	-	992 B
Evict_TwoLayers	1	1,087.1 ns	20.38 ns	18.07 ns	0.5608	-	-	1760 B
Cleanup	1	1,134.3 ns	19.59 ns	18.32 ns	0.3567	-	-	1120 B
Cleanup_TwoLayers	1	1,486.1 ns	27.56 ns	25.78 ns	0.6008	-	-	1888 B
GetMiss	1	411.9 ns	2.69 ns	2.25 ns	0.2904	-	-	912 B
GetHit	1	767.4 ns	13.18 ns	15.69 ns	0.3157	-	-	992 B
GetOrSet_NeverStale	1	1,316.5 ns	23.25 ns	21.75 ns	0.4196	-	-	1320 B
GetOrSet_AlwaysStale	1	1,318.2 ns	21.61 ns	20.21 ns	0.4196	-	-	1320 B
GetOrSet_TwoSimultaneous	1	32,743,879.1 ns	632,491.11 ns	728,377.70 ns	-	-	-	2448 B
GetOrSet_FourSimultaneous	1	32,739,752.6 ns	635,172.47 ns	705,992.52 ns	-	-	-	2592 B
SetupAndTeardown	100	314.2 ns	6.07 ns	6.24 ns	0.2904	-	-	912 B
Set	100	32,036.8 ns	583.86 ns	695.04 ns	1.2817	-	-	4160 B
Set_TwoLayers	100	39,942.5 ns	561.38 ns	525.11 ns	1.5259	-	-	4928 B
Evict	100	44,376.5 ns	756.07 ns	707.23 ns	2.8076	-	-	8912 B
Evict_TwoLayers	100	58,296.5 ns	942.72 ns	925.87 ns	4.5776	-	-	14432 B
Cleanup	100	62,802.8 ns	1,188.87 ns	1,112.07 ns	12.2070	-	-	38576 B
Cleanup_TwoLayers	100	102,710.7 ns	1,902.99 ns	1,589.08 ns	5.6152	-	-	17728 B
GetMiss	100	9,375.3 ns	170.97 ns	159.93 ns	0.2899	-	-	912 B
GetHit	100	10,189.7 ns	91.90 ns	85.97 ns	0.3052	-	-	992 B
GetOrSet_NeverStale	100	26,890.5 ns	425.45 ns	397.97 ns	0.3967	-	-	1320 B
GetOrSet_AlwaysStale	100	101,666.5 ns	1,856.87 ns	1,736.92 ns	7.4463	-	-	23496 B
GetOrSet_TwoSimultaneous	100	32,583,572.7 ns	621,719.48 ns	691,039.56 ns	-	-	-	17072 B
GetOrSet_FourSimultaneous	100	33,194,950.8 ns	655,316.61 ns	1,130,388.96 ns	-	-	-	28304 B

MemoryCacheLayer Benchmark

Method	WorkIterations	Mean [ns]	Error [ns]	StdDev [ns]	Gen 0	Gen 1	Gen 2	Allocated [B]
Overhead	1	199.1 ns	3.95 ns	4.06 ns	0.2270	-	-	712 B
GetMiss	1	252.7 ns	4.48 ns	4.19 ns	0.2270	-	-	712 B
GetHit	1	498.6 ns	8.26 ns	7.73 ns	0.2518	-	-	792 B
SetExisting	1	717.7 ns	11.96 ns	16.76 ns	0.2623	-	-	824 B
EvictMiss	1	267.6 ns	4.71 ns	4.40 ns	0.2270	-	-	712 B
EvictHit	1	519.9 ns	9.55 ns	8.93 ns	0.2518	-	-	792 B
Cleanup	1	948.5 ns	11.03 ns	9.78 ns	0.2918	-	-	920 B
Overhead	100	196.2 ns	2.85 ns	2.67 ns	0.2270	-	-	712 B
GetMiss	100	3,432.4 ns	45.35 ns	42.42 ns	0.2251	-	-	712 B
GetHit	100	4,165.4 ns	78.51 ns	73.43 ns	0.2518	-	-	792 B
SetExisting	100	24,103.2 ns	428.71 ns	401.01 ns	1.2512	-	-	3992 B
EvictMiss	100	5,516.0 ns	91.52 ns	85.61 ns	0.2213	-	-	712 B
EvictHit	100	31,023.4 ns	598.90 ns	588.20 ns	2.7466	-	-	8712 B
Cleanup	100	42,737.4 ns	545.33 ns	510.10 ns	8.6670	-	-	27328 B

[Turnerj]

The performance difference seems to be somewhat inconclusive. There are perhaps minor drawbacks for direct memory layer benchmarks around GetHit and Cleanup however as a trade for code simplicity, that doesn't seem too bad.