[Validation Approach] Improve ConcurrentStack.TryPopRange performance under contention.

[aw0lid]

Overview

This PR is opened as a validation PR to investigate potential approaches for improving ConcurrentStack<T>.TryPopRange performance under contention, as described in #100083.

The current implementation attempts to pop the requested number of items as a single batch. Under contention, repeated CAS failures can cause the same large portion of the stack to be traversed repeatedly, leading to a significant performance degradation.

Approach 1: Adaptive Batch Sizing

The first approach explored in this PR is adaptive batch sizing.
The batch size starts at the requested count.
If the CAS fails 30 times consecutively, the batch size is reduced by half and the operation is retried. After a successful CAS, the popped batch is copied to the destination array and the remaining items are processed using the current batch size.
If contention increases again and the CAS fails another 30 times, the batch size is reduced again.

For example:

Requested: 1,000,000
Current batch: 1,000,000

        CAS fails ×30
              ↓
Current batch: 500,000

        CAS fails ×30
              ↓
Current batch: 250,000

        CAS succeeds
              ↓
       Pop 250,000
       Remaining: 750,000
              ↓
Next batch: 250,000

        CAS succeeds
              ↓
       Pop 250,000
       Remaining: 500,000
              ↓
Next batch: 250,000

        CAS fails ×30
              ↓
Current batch: 125,000

        CAS succeeds
              ↓
       Pop 125,000
       Remaining: 375,000
              ↓
Next batch: 125,000
              ...
              ↓
Remaining: 0
              ↓
          RETURN 1M

The goal of this approach is to avoid repeatedly traversing a large number of nodes when the stack is highly contended, while still allowing large batches when contention is low.

Initial results

Show benchmark code

public class ConcurrentStackBenchmarks
{
    private readonly ConcurrentStack<int> _stack = new();
    private int[] _popRangeBuffer;
    private bool _running = true;

    [Params(1_000_000)]
    public int TryPopCount;

    [Params(0, 1, 2, 3, 4, 5, 6, 7, 8)]
    public int ParallelThreads;

    [GlobalSetup]
    public void Setup()
    {
        _popRangeBuffer = new int[TryPopCount];

        for (int i = 0; i < TryPopCount; i++)
        {
            _stack.Push(i);
        }

        for (int i = 0; i < ParallelThreads; i++)
        {
            _ = Task.Run(() =>
            {
                while (Volatile.Read(ref _running))
                {
                    _stack.Push(42);
                    _stack.TryPop(out _);
                }
            });
        }
    }

    [GlobalCleanup]
    public void Cleanup() => _running = false;

    [Benchmark]
    public void PopRange()
    {
        int popped = _stack.TryPopRange(_popRangeBuffer);
        if (popped != TryPopCount) throw new Exception("?");
        _stack.PushRange(_popRangeBuffer);
    }
}

Parallel Threads	Mean Before	Mean After	Performance	Allocated Before	Allocated After	Allocation
0	97.08 ms	88.00 ms	-9.4%	30.52 MB	30.52 MB	0%
1	136.92 ms	125.07 ms	-8.7%	52.32 MB	48.39 MB	-7.5%
2	251.16 ms	210.01 ms	-16.4%	86.83 MB	76.98 MB	-11.3%
3	2,056.74 ms	429.75 ms	-79.1%	647.28 MB	247.48 MB	-61.8%
4	11,217.58 ms	470.94 ms	-95.8%	590.65 MB	259.91 MB	-56.0%
5	24,535.96 ms	465.93 ms	-98.1%	567.47 MB	245.29 MB	-56.8%
6	1,065.28 ms	466.86 ms	-56.2%	148.70 MB	255.24 MB	+71.6%
7	10,673.27 ms	477.15 ms	-95.5%	9,368.18 MB	249.82 MB	-97.3%
8	58,015.97 ms	475.71 ms	-99.2%	15,822.80 MB	268.41 MB	-98.3%

The most significant improvement can be seen under high contention, where the benchmark goes from ~58 seconds to ~476 ms with 8 threads, while allocations drop from ~15.8 GB to ~268 MB.

These results are encouraging and show that adaptive batch sizing can significantly reduce the performance degradation under contention.

Further approaches may be explored in this PR to evaluate different strategies and trade-offs.

Open Question

One important aspect that still needs to be evaluated is whether performing multiple successful CAS operations within a single TryPopRange invocation is compatible with the atomicity semantics expected from the API.

Further validation and discussion are needed before considering any approach as a final implementation.

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-collections
See info in area-owners.md if you want to be subscribed.

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.