Throughput analysis v2.0

[bitfaster]

• Use BenchmarkDotNet's method to keep return values alive, mitigating dead code elimination for the return value from cache GetOrAdd. This reduces peak throughput by about 10%, likely due to the extra non-inlined method.
• Apply optimization option to test execution method.
• Use Threads instead of Tasks. This is more stable, but MemoryCache results now have more extreme outliers due to unpredictable nature of background thread eviction.
• Replicate the BenchmarkDotNet main loop to determine when the result is stable, based on computing statistics from the results so far.
• Use Perfolizer to eliminate outliers from the result calculation - this eliminates some of the outliers from MemoryCache when the eviction thread is not scheduled. The update and evict scenarios provoke unbounded growth/failure to evict for MemoryCache due to threadpool backlog.
• Adapt the number of iterations in each run so that a run is approximately 200ms. This gives a worst-case scenario time of about 35 secs, so an entire test run can complete on a 96 core machine in less than 1 day.
• Wait for the thread pool to empty before each run. This flushes out any backlog that MemoryCache has built up, making results more consistent.

---

Results for adaptive mode:

Min iters	Max iters	Max Relative Error	Target Iteration time
15	80	0.02	200ms

[coveralls]

coverage: 99.159% (+0.06%) from 99.1%
when pulling ac3b935 on users/alexpeck/deadcode
into f853da2 on main.

[bitfaster]

---

Results for:

Min iters	Max iters	Max Relative Error
6	25	0.04

---

Results for BenchmarkDotNet settings (now significantly slower for eviction test):

Min iters	Max iters	Max Relative Error
15	80	0.02