[API Proposal]: Improve RateLimiter metrics

[BrennanConroy]

Background and motivation

One of the original potential use cases cited for RateLimiter.GetAvailablePermits() was for diagnostics. While it's a good starting point, it would make more sense to include more information to include in diagnostics. So, taking inspiration from MemoryCache we are proposing changing GetAvailablePermits to a more useful API: GetStatistics.

API Proposal

namespace System.Threading.RateLimiting;

public abstract class RateLimiter : IAsyncDisposable, IDisposable
{
-    public abstract int GetAvailablePermits();
+    public abstract RateLimiterStatistics? GetStatistics();
}

public abstract class PartitionedRateLimiter<TResource> : IAsyncDisposable, IDisposable
{
-    public abstract int GetAvailablePermits(TResource resourceID);
+    public abstract RateLimiterStatistics? GetStatistics(TResource resourceID);
}

+ public class RateLimiterStatistics
+ {
+    public RateLimiterStatistics();

+    public long CurrentAvailablePermits { get; init; }

+    public long CurrentQueuedCount { get; init; }

+    public long TotalFailedLeases { get; init; }

+    public long TotalSuccessfulLeases { get; init; }
+ }

API Usage

Meter meter = new Meter("RateLimiter", "1.0.0");
RateLimiter limiter = new ConcurrencyLimiter(
    new ConcurrencyLimiterOptions(100, QueueProcessingOrder.OldestFirst, 20));

meter.CreateObservableGauge<long>("available-permits", GetAvailablePermits);

IEnumerable<Measurement<long>> GetAvailablePermits()
{
    return new Measurement<long>[]
    {
        new Measurement<long>(limiter.GetStatistics()!.CurrentAvailablePermits,
            new KeyValuePair<string, object?>("Limiter", "Concurrent")),
    };
}

Alternative Designs

An alternative name is GetCurrentStatistics which is what MemoryCache uses.

Another property that might be useful to include is long TotalAcquiredPermits { get; init; }

There has been discussion around allowing rate limiter specific state to be returned, either via this API or another API. This would be similar to HttpClientHandler.Properties

runtime/src/libraries/System.Net.Http/src/System/Net/Http/HttpClientHandler.cs

Line 300 in 8cd9d63

public IDictionary<string, object?> Properties => _underlyingHandler.Properties;

Two ways to allow state without introducing additional APIs to what is already proposed is to use the existing metadata on RateLimitLease and call Acquire with 0 permits to always get back a lease without acquiring any permits. Or we keep the new RateLimiterStatistics class that's being proposed as non-sealed and allow implementations to provide their own statistics type that contains custom state.

Risks

If someone was wanting to query GetAvailablePermits before every call to Acquire or WaitAsync they would now need to call a more expensive method (allocates every call) to get the same behavior. I don't believe this would be a common pattern, if someone wants to acquire permits quickly without potential waiting, they can call Acquire which will already check for availability and be more accurate than calling GetAvailablePermits then Acquire if there are enough permits.

[ghost]

Tagging subscribers to this area: @mangod9
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Background and motivation

One of the original potential use cases cited for RateLimiter.GetAvailablePermits() was for diagnostics. While it's a good starting point, it would make more sense to include more information to include in diagnostics. So, taking inspiration from MemoryCache we are proposing changing GetAvailablePermits to a more useful API: GetStatistics.

API Proposal

namespace System.Threading.RateLimiting;

public abstract class RateLimiter : IAsyncDisposable, IDisposable
{
-    public abstract int GetAvailablePermits();
+    public abstract RateLimiterStatistics? GetStatistics();
}

public abstract class PartitionedRateLimiter<TResource> : IAsyncDisposable, IDisposable
{
-    public abstract int GetAvailablePermits(TResource resourceID);
+    public abstract RateLimiterStatistics? GetStatistics(TResource resourceID);
}

+ public class RateLimiterStatistics
+ {
+    public RateLimiterStatistics();

+    public long CurrentAvailablePermits { get; init; }

+    public long CurrentQueuedCount { get; init; }

+    public long TotalFailedLeases { get; init; }

+    public long TotalSuccessfulLeases { get; init; }
+ }

API Usage

Meter meter = new Meter("RateLimiter", "1.0.0");
RateLimiter limiter = new ConcurrencyLimiter(
    new ConcurrencyLimiterOptions(100, QueueProcessingOrder.OldestFirst, 20));

meter.CreateObservableGauge<long>("available-permits", GetAvailablePermits);

IEnumerable<Measurement<long>> GetAvailablePermits()
{
    return new Measurement<long>[]
    {
        new Measurement<long>(limiter.GetStatistics()!.CurrentAvailablePermits,
            new KeyValuePair<string, object?>("Limiter", "Concurrent")),
    };
}

Alternative Designs

An alternative name is GetCurrentStatistics which is what MemoryCache uses.

Another property that might be useful to include is long TotalAcquiredPermits { get; init; }

There has been discussion around allowing rate limiter specific state to be returned, either via this API or another API. This would be similar to HttpClientHandler.Properties

runtime/src/libraries/System.Net.Http/src/System/Net/Http/HttpClientHandler.cs

Line 300 in 8cd9d63

public IDictionary<string, object?> Properties => _underlyingHandler.Properties;

Two ways to allow state without introducing additional APIs to what is already proposed is to use the existing metadata on RateLimitLease and call Acquire with 0 permits to always get back a lease without acquiring any permits. Or we keep the new RateLimiterStatistics class that's being proposed as non-sealed and allow implementations to provide their own statistics type that contains custom state.

Risks

If someone was wanting to query GetAvailablePermits before every call to Acquire or WaitAsync they would now need to call a more expensive method (allocates every call) to get the same behavior. I don't believe this would be a common pattern, if someone wants to acquire permits quickly without potential waiting, they can call Acquire which will already check for availability and be more accurate than calling GetAvailablePermits then Acquire if there are enough permits.

Author:	BrennanConroy
Assignees:	-
Labels:	api-suggestion, area-System.Threading
Milestone:	-

[bartonjs]

Video

Looks good as proposed. (Changed resourceID to resource to match an in-flight rename request)

namespace System.Threading.RateLimiting;

public abstract class RateLimiter : IAsyncDisposable, IDisposable
{
-    public abstract int GetAvailablePermits();
+    public abstract RateLimiterStatistics? GetStatistics();
}

public abstract class PartitionedRateLimiter<TResource> : IAsyncDisposable, IDisposable
{
-    public abstract int GetAvailablePermits(TResource resourceID);
+    public abstract RateLimiterStatistics? GetStatistics(TResource resource);
}

+ public class RateLimiterStatistics
+ {
+    public RateLimiterStatistics();

+    public long CurrentAvailablePermits { get; init; }

+    public long CurrentQueuedCount { get; init; }

+    public long TotalFailedLeases { get; init; }

+    public long TotalSuccessfulLeases { get; init; }
+ }