Revise LRU eviction logic

[bitfaster]

• Simplify eviction logic: remove retry loop and simply discard the coldest item. Logic is now entirely based on the result of the preceding interlocked operation, so now there are no races.
• Inline helper methods
• Consolidate removal logic to always call into OnRemove

There is a marginal improvement in throughput for the eviction scenario for higher thread counts, likely because some volatile read calls have been eliminated.

MaxCycles = 3 was chosen because that is the threshold where the Arc OLTP scenario is closest to optimal. There is almost no difference in other scenarios.

[coveralls]

coverage: 98.254% (-0.3%) from 98.581%
when pulling 006bb46 on users/alexpeck/evictthru
into bebba07 on main.

[bitfaster]

Wikipedia hit rate:

CacheSize	Lru	MemoryCache	ConcurrentLru	ConcurrentLfu
25	12.339894258281078	13.443190612152927	27.106598984771573	28.86727297032638
50	21.441939220308402	22.77921819896573	36.238106955426304	37.18232134748054
75	27.84286871100062	29.020668814532556	40.60856398437633	40.98620692807719
100	32.25609649363885	32.87770956047091	43.25343594805921	43.305737080834184
125	35.30941454952625	35.79993919198931	45.37747514748507	45.033543076744856
150	37.484624347754924	37.863226126974574	46.86929834312107	46.60584379030802
175	39.10013286561485	39.37033048496005	48.16184652990179	48.25709457451841
200	40.35904822418645	40.49178004803766	49.136933195811054	48.46133322957098

Glimpse hit rate:

CacheSize	Lru	MemoryCache	ConcurrentLru	ConcurrentLfu
250	0.914380714879468	8.246051537822112	12.784704904405652	15.181243764549384
500	0.9476309226932668	19.334995843724023	26.08478802992519	33.03363303363304
750	1.14713216957606	33.43308395677473	39.38487115544472	43.34942567005161
1000	11.205320033250207	45.18703241895262	43.275145469659186	50.61666666666667
1250	21.24688279301746	49.24355777223607	49.925187032418954	51.675279213202195
1500	36.55860349127182	53.36658354114713	52.40232751454697	53.37556259376562
1750	45.03740648379053	54.54696591853699	53.49958437240233	55.40180060020007
2000	57.40648379052369	57.40648379052369	53.54945968412302	57.800767562155855

Arc Database

CacheSize	ClassicLruHitRate	MemoryCacheHitRate	ConcurrentLruHitRate	ConcurrentLfuHitRate
1000000	NaN	NaN	11.710457520183226	NaN
2000000	NaN	NaN	23.083234978788113	NaN
3000000	NaN	NaN	36.2131806538564	NaN
4000000	NaN	NaN	39.46113553789832	NaN
5000000	NaN	NaN	44.934996994278386	NaN
6000000	NaN	NaN	50.37551671172294	NaN
7000000	NaN	NaN	55.99905676650708	NaN
8000000	NaN	NaN	60.92242716785198	NaN

Arc Search

CacheSize	ClassicLruHitRate	MemoryCacheHitRate	ConcurrentLruHitRate	ConcurrentLfuHitRate
100000	NaN	NaN	8.772008414097234	NaN
200000	NaN	NaN	17.29688289073022	NaN
300000	NaN	NaN	25.03243903381473	NaN
400000	NaN	NaN	32.25750930873805	NaN
500000	NaN	NaN	39.141495865783035	NaN
600000	NaN	NaN	45.86026733055001	NaN
700000	NaN	NaN	52.051201319965465	NaN
800000	NaN	NaN	57.583828619022945	NaN

Arc OLTP

This one is a significant regression, putting ConcurrentLRU bottom across all sizes.

CacheSize	ClassicLruHitRate	MemoryCacheHitRate	ConcurrentLruHitRate	ConcurrentLfuHitRate
250	NaN	NaN	9.04309491382658	NaN
500	NaN	NaN	19.837990690754747	NaN
750	NaN	NaN	25.987452756400792	NaN
1000	NaN	NaN	30.079582560753494	NaN
1250	NaN	NaN	33.13270870594928	NaN
1500	NaN	NaN	35.64609553189045	NaN
1750	NaN	NaN	37.47917452920488	NaN
2000	NaN	NaN	39.46288608481149	NaN

[bitfaster]

Arc OLTP with 3 cycle attempts

CacheSize	ClassicLruHitRate	MemoryCacheHitRate	ConcurrentLruHitRate	ConcurrentLfuHitRate
250	NaN	NaN	12.931318335712605	NaN
500	NaN	NaN	25.05926302720028	NaN
750	NaN	NaN	30.614946206564603	NaN
1000	NaN	NaN	34.495402808088436	NaN
1250	NaN	NaN	37.24409147345333	NaN
1500	NaN	NaN	39.6604477407851	NaN
1750	NaN	NaN	41.64722226780215	NaN
2000	NaN	NaN	43.57306554211859	NaN