Make RateLimiterMiddleware endpoint-aware

[wtgodbe]

Allow users to piecemeal add named RateLimiters that apply to specific endpoints - will achieve this w/ IEndpointConventionBuilder. Some discussion starting at #41655 (comment)

namespace Microsoft.AspNetCore.RateLimiting
{
    public interface IRateLimiterPolicy<TKey>
    {
        public int CustomRejectionStatusCode { get; }
        public RateLimitPartition<TKey> GetPartition(HttpContext httpContext);
    }

    public sealed class RateLimiterOptions
    {
        public PartitionedRateLimiter<HttpContext> Limiter { get; set; }
        public Func<HttpContext, RateLimitLease, Task> OnRejected { get; set; }
        public int DefaultRejectionStatusCode{ get; set; }
        public RateLimiterOptions AddPolicy<TKey>(string name, Func<HttpContext, RateLimitPartition<TKey>> partitioner, bool global = false)
        public RateLimiterOptions AddPolicy<TKey, TPolicy>(string name, bool global = false) where TPolicy : IRateLimiterPolicy<TKey>
    }

    public static class RateLimiterApplicationBuilderExtensions
    {
        public static IApplicationBuilder UseRateLimiter(this IApplicationBuilder app)
        public static IApplicationBuilder UseRateLimiter(this IApplicationBuilder app, RateLimiterOptions options)
    }

    public static class RateLimiterEndpointConventionBuilderExtensions
    {
        public static TBuilder RequireRateLimiting<TBuilder>(this TBuilder builder, String name) where TBuilder : IEndpointConventionBuilder
    }

    public static class RateLimiterOptionsExtensions
    {
        public static RateLimiterOptions AddTokenBucketRateLimiter(this RateLimiterOptions options, string name, TokenBucketRateLimiterOptions tokenBucketRateLimiterOptions, bool global = false)
        public static RateLimiterOptions AddFixedWindowRateLimiter(this RateLimiterOptions options, string name, FixedWindowRateLimiterOptions fixedWindowRateLimiterOptions, bool global = false)
        public static RateLimiterOptions AddSlidingWindowRateLimiter(this RateLimiterOptions options, string name, SlidingWindowRateLimiterOptions slidingWindowRateLimiterOptions, bool global = false)
        public static RateLimiterOptions AddConcurrencyLimiter(this RateLimiterOptions options, string name, ConcurrencyLimiterOptions concurrencyLimiterOptions, bool global = false)
        public static RateLimiterOptions AddNoLimiter(this RateLimiterOptions options, string name, bool global = false)
    }
}

[Kahbazi]

Is global = true equivalent to default policy like what there is in Cors?

aspnetcore/src/Middleware/CORS/src/Infrastructure/CorsOptions.cs

Line 34 in ae281e9

public void AddDefaultPolicy(CorsPolicy policy)

If not a default policy would be useful. Also there could be a NoRateLimiting method on endpoint too to skip the middleware completely.

[wtgodbe]

Is global = true equivalent to default policy like what there is in Cors?

global = true means the limiter can be shared across endpoints that request it, not that it will be shared across every endpoint. The default is false to avoid users accidentally supplying 2 limiter policies with the same name across 2 different endpoints - we want the default behavior to be that that is 2 separate limiters.

There's not currently a way w/ the runtime APIs to mix endpoint-specific & truly global limiters - if you want a truly global limiter, you can set the Limiter on RateLimiterOptions.

If not a default policy would be useful. Also there could be a NoRateLimiting method on endpoint too to skip the middleware completely.

Good call, just added an extension method for AddNoLimiter

[wtgodbe]

Notes - name params should be policyName
Inline policies #39840 should be in for 7, not necessarily needed for this PR
bool global should be PolicyScope enum, which we can document

• Actually, don't have global at all - Limiters can not be shared across policies (if 2 policies have same key, we'll disambiguate). Users can apply the same policy to multiple endpoints, which will share limiters across endpoints

Only 1 policy metadata will be resolved on each endpoint - last one wins (imagine applying policy to whole group, then different policy to 1 endpoint in group). Check the endpoint's metadata, see if we have any policy in our last matching that.

Follow-up - allow user to set feature on context that has policy name. Call to .Create checks feature, then endpoint metadata

[ghost]

Thank you for submitting this for API review. This will be reviewed by @dotnet/aspnet-api-review at the next meeting of the ASP.NET Core API Review group. Please ensure you take a look at the API review process documentation and ensure that:

• The PR contains changes to the reference-assembly that describe the API change. Or, you have included a snippet of reference-assembly-style code that illustrates the API change.
• The PR describes the impact to users, both positive (useful new APIs) and negative (breaking changes).
• Someone is assigned to "champion" this change in the meeting, and they understand the impact and design of the change.

[halter73]

API Review Notes:

• We need to see some samples.
• This should be a diff so we can see what API is new vs which API exists.
• Is there a way to define a policy inline for the rate limiter?
  • Not yet. See Allow direct configuration of authorization policies via endpoint metadata #39840 for an auth example.
  • This can be done in a later API review if we feel this is worthwhile
• In AddTokenBucketRateLimiter and similar, the string name parameter should be a string policyName parameter.
  • All string name parameters should be string policyName.
• What about "global = false" what is that about?

  services.AddLimiter("myRolePolicy", context =>
  {
      if (context.IsAdmin()) // Pretend this extension method is defined
      {
          return RateLimitPartion.CreateNoLimiter("admin");
      }
      else
      {
          return RateLimitPartion.CreateTokenBucketLimiter("pleb", //...
      }
  });
  
  services.AddLimiter("myUserNamePolicy", context =>
      RateLimitPartion.Create(httpContext.User.Identity.Name, //...

  • Non-global should be the only option. We don't think it's realistic for different policies to want to share RateLimitPartionKey.
• Do we want to be able to specify OnRejected per policy? We can already set CustomRejectionStatusCode with IRateLimiterPolicy.
  • Didn't get to this.
• Can RateLimiterOptions.Limiter be merged with the endpoint-specific stuff? If you don't want to use the endpoint-specific stuff, you just don't define policies or call RequireRateLimiting.
• Can I use a something on the HttpContext to select a policy name if the request is not hitting an endpoint?
  • Maybe a feature would let you set a string? RateLimterPolicyName. Who would set the feature considering middleware hasn't run yet? We had a similar issue with output caching cc @sebastienros
  • Maybe we could have an Func<HttpContext, string?> RateLimiterOptions.PolicyNameCallback

[halter73]

@BrennanConroy @wtgodbe Do you have any idea about what kind of rate limiter chaining APIs we would like to propose in the runtime repo?

[wtgodbe]

namespace Microsoft.AspNetCore.RateLimiting
{
+   public interface IRateLimiterPolicy<TKey>
+   {
+       public int CustomRejectionStatusCode { get; }
+       public Func<HttpContext, RateLimitLease, Task> CustomOnRejected { get; }
+       public RateLimitPartition<TKey> GetPartition(HttpContext httpContext);
+   }

    public sealed class RateLimiterOptions
    {
        public PartitionedRateLimiter<HttpContext> Limiter { get; set; }
        public Func<HttpContext, RateLimitLease, Task> OnRejected { get; set; }
        public int DefaultRejectionStatusCode{ get; set; }
+       public RateLimiterOptions AddPolicy<TKey>(string policyName, Func<HttpContext, RateLimitPartition<TKey>> partitioner)
+       public RateLimiterOptions AddPolicy<TKey, TPolicy>(string policyName) where TPolicy : IRateLimiterPolicy<TKey>
    }

-   public static class RateLimitingApplicationBuilderExtensions
+   public static class RateLimiterApplicationBuilderExtensions
    {
        public static IApplicationBuilder UseRateLimiter(this IApplicationBuilder app)
        public static IApplicationBuilder UseRateLimiter(this IApplicationBuilder app, RateLimiterOptions options)
    }

+   public static class RateLimiterEndpointConventionBuilderExtensions
+   {
+       public static TBuilder RequireRateLimiting<TBuilder>(this TBuilder builder, String policyName) where TBuilder : IEndpointConventionBuilder
+   }

+   public static class RateLimiterOptionsExtensions
+   {
+       public static RateLimiterOptions AddTokenBucketRateLimiter(this RateLimiterOptions options, string policyName, TokenBucketRateLimiterOptions tokenBucketRateLimiterOptions)
+       public static RateLimiterOptions AddFixedWindowRateLimiter(this RateLimiterOptions options, string policyName, FixedWindowRateLimiterOptions fixedWindowRateLimiterOptions)
+       public static RateLimiterOptions AddSlidingWindowRateLimiter(this RateLimiterOptions options, string policyName, SlidingWindowRateLimiterOptions slidingWindowRateLimiterOptions)
+       public static RateLimiterOptions AddConcurrencyLimiter(this RateLimiterOptions options, string policyName, ConcurrencyLimiterOptions concurrencyLimiterOptions)
+       public static RateLimiterOptions AddNoLimiter(this RateLimiterOptions options, string policyName)
+   }
}

[wtgodbe]

Sample:

var echoPolicyName = "echoPolicy";
var echo2PolicyName = "echoPolicy2";

var builder = WebApplication.CreateBuilder(args);

builder.UseRateLimiter(options =>
{
    options.AddConcurrencyLimiter(echoPolicyName,
                new ConcurrencyLimiterOptions(2, QueueProcessingOrder.OldestFirst, 3));

    options.AddPolicy<string>(echo2PolicyName, context =>
            if (context.User.Identity.Name == "admin")
            {
                return RateLimitPartition.CreateTokenBucketLimiter("AdminLimiter",
                    _ => new TokenBucketRateLimiterOptions(5, QueueProcessingOrder.OldestFirst, 5, new TimeSpan(0, 1, 0), 3));
            }
            else
            {
                return RateLimitPartition.CreateNoLimiter("DefaultLimiter");
            });
});

var app = builder.Build();

app.UseEndpoints(endpoints =>
{
    endpoints.MapGet("/echo",
        context => context.Response.WriteAsync("echo"))
        .RequireRateLimiting(echoPolicyName);

    endpoints.MapGet("/echo2",
        context => context.Response.WriteAsync("echo2"))
        .RequireRateLimiting(echo2PolicyName);

    endpoints.MapRazorPages();
});

app.Run();

[davidfowl]

@wtgodbe Can you show a controller as well?

[halter73]

Can you show a controller as well?

That would require an attribute. We could make it generic! I'm personally okay doing that in a later PR, but we should know it's possible at least.

return RateLimitPartition.CreateTokenBucketLimiter("AdminLimiter",

I like how it's the admin that gets rate limited in this sample! 😆

[ghost]

Thank you for submitting this for API review. This will be reviewed by @dotnet/aspnet-api-review at the next meeting of the ASP.NET Core API Review group. Please ensure you take a look at the API review process documentation and ensure that:

• The PR contains changes to the reference-assembly that describe the API change. Or, you have included a snippet of reference-assembly-style code that illustrates the API change.
• The PR describes the impact to users, both positive (useful new APIs) and negative (breaking changes).
• Someone is assigned to "champion" this change in the meeting, and they understand the impact and design of the change.

[BrennanConroy]

And a sample with IRateLimiterPolicy please, since I was definitely confused about it when we first reviewed the API.

[wtgodbe]

Sample:

public class MyRateLimiterPolicy<string> : IRateLimiterPolicy<string>
{
    public Func<HttpContext, RateLimitLease, Task> CustomOnRejected
    {
        get => (context, lease) =>
        {
            return Task.CompletedTask;
        };
    }

    public int CustomRejectionStatusCode
    {
        get => 529;
    }

    public RateLimitPartition<string> GetPartition(HttpContext context)
    {
        if (context.User.Identity.Name == "admin")
        {
            return RateLimitPartition.CreateTokenBucketLimiter("AdminLimiter",
                _ => new TokenBucketRateLimiterOptions(5, QueueProcessingOrder.OldestFirst, 5, new TimeSpan(0, 1, 0), 3));
        }
        else
        {
            return RateLimitPartition.CreateNoLimiter("DefaultLimiter");
        });
    }
}

var echoPolicyName = "echoPolicy";

var builder = WebApplication.CreateBuilder(args);

builder.UseRateLimiter(options =>
{
    options.AddPolicy<string, MyRateLimiterPolicy<string>>(echoPolicyName);
});

var app = builder.Build();

app.UseEndpoints(endpoints =>
{
    endpoints.MapGet("/echo",
        context => context.Response.WriteAsync("echo"))
        .RequireRateLimiting(echoPolicyName);

    endpoints.MapRazorPages();
});

app.Run();

[halter73]

API Review Notes:

• RateLimitPartition.CreateNoLimiter() Makes it sound like a limiter or partition is being created every time the method is called.
  • This is already approved runtime API, but it should change.
  • We should communicate this is not creating a limiter or a partition necessarily. Most of the time it's just crating a key to look up an existing partition. It also happens to include information about how to create a partition if need be, but this happens less frequently.
  • We should submit a runtime API proposal for this. RateLimitPartition.Choose()?
• Do we like "Custom" in CustomRejectionStatusCode and CustomOnRejected?
  • Should the "Custom" properties be nullable?
  • Yes to both.
• Should DefaultRejectionStatusCode be RejectionStatusCode?
  • Yes.
• Can we just get rid of IRateLimiterPolicy<TKey>.RejectionStatusCode.
• I can inject DI services into MyRateLimiterPolicy's constructors right?
• AddTokenBucketRateLimiter -> AddTokenBucketLimiter
  • Remove Rate from all of these.
• Should HttpContext, RateLimitLease be added to a context class for future proofing
  • OnRejectedContext class?
  • Yes. Let's use required init here too.
• IRateLimitMetadata (not in the proposal currently, but in the PR) should be made internal for now.
• Task to ValueTask.
• PartitionedRateLimiter<HttpContext> Limiter to PartitionedRateLimiter<HttpContext>? GlobalLimiter
• Add CancellationToken
• TKey -> TPartitionKey

namespace Microsoft.AspNetCore.RateLimiting
{
-  public interface IRateLimitMetadata
- {
- }
-
+   public interface IRateLimiterPolicy<TPartitionKey>
+   {
+       public Func<OnRejectedContext, CancellationToken, ValueTask>? OnRejected { get; }
+       public RateLimitPartition<TPartitionKey> GetPartition(HttpContext httpContext);
+   }

    public sealed class RateLimiterOptions
    {
-        public PartitionedRateLimiter<HttpContext> Limiter { get; set; }
+        public PartitionedRateLimiter<HttpContext>? GlobalLimiter { get; set; }
-        public Func<HttpContext, RateLimitLease, Task> OnRejected { get; set; }
+        public Func<OnRejectedContext, CancellationToken, ValueTask>? OnRejected { get; set; }
-        public int DefaultRejectionStatusCode { get; set; }
+        public int RejectionStatusCode { get; set; }
+        public RateLimiterOptions AddPolicy<TPartitionKey>(string policyName, Func<HttpContext, RateLimitPartition<TPartitionKey>> partitioner)
+        public RateLimiterOptions AddPolicy<TPartitionKey, TPolicy>(string policyName) where TPolicy : IRateLimiterPolicy<TPartitionKey>
+        public RateLimiterOptions AddPolicy<TPartitionKey>(string policyName, IRateLimiterPolicy<TPartitionKey> policy);
    }

+ // We could add the policy name to this in the future if we want.
+ public sealed class OnRejectedContext
+ {
+       public HttpContext HttpContext { get; required init; }
+       public RateLimitLease Lease { get; required init; }
+ }
+
+   public static class RateLimiterOptionsExtensions
+   {
+       public static RateLimiterOptions AddTokenBucketLimiter(this RateLimiterOptions options, string policyName, TokenBucketRateLimiterOptions tokenBucketRateLimiterOptions)
+       public static RateLimiterOptions AddFixedWindowLimiter(this RateLimiterOptions options, string policyName, FixedWindowRateLimiterOptions fixedWindowRateLimiterOptions)
+       public static RateLimiterOptions AddSlidingWindowLimiter(this RateLimiterOptions options, string policyName, SlidingWindowRateLimiterOptions slidingWindowRateLimiterOptions)
+       public static RateLimiterOptions AddConcurrencyLimiter(this RateLimiterOptions options, string policyName, ConcurrencyLimiterOptions concurrencyLimiterOptions)
+       public static RateLimiterOptions AddNoLimiter(this RateLimiterOptions options, string policyName)
+   }

-   public static class RateLimitingApplicationBuilderExtensions
-   {
-       //...
-   }
}
+ namespace  Microsoft.AspNetCore.Builder
+ {
+   public static class RateLimitingApplicationBuilderExtensions
+ {
+     //...
+ }
+
+   public static class RateLimiterEndpointConventionBuilderExtensions
+   {
+       public static TBuilder RequireRateLimiting<TBuilder>(this TBuilder builder, string policyName) where TBuilder : IEndpointConventionBuilder
+   }
}

API Approved!

[Kahbazi]

How about a SkipRateLimiting method on IEndpointConventionBuilder beside AddNoLimiter.

+   public static class RateLimiterEndpointConventionBuilderExtensions
+   {
+       public static TBuilder SkipRateLimiting<TBuilder>(this TBuilder builder) where TBuilder : IEndpointConventionBuilder
+   }

This way when executing endpoints marked with SkipRateLimiting there's no need to look up for policies and the middleware could be skipped completely.

[wtgodbe]

This is done