About Ocelot's advanced Rate Limiting functionality

[ms30063600]

Hi,

I have the following question.
As per the documentation, rate limiting in Ocelot can be configured for each route, and that following parameters apply:

• Period: The time window in which the limit applies. Supported formats include 1s (seconds), 5m (minutes), 1h (hours), or 1d (days).
• Limit: The maximum number of requests allowed within the defined Period.
• PeriodTimespan: The duration in seconds the client must wait before retrying after hitting the quota limit.

My aim is to limit the use of a specific route that contains a dynamic placeholder. For example consider "UpstreamPathTemplate": "/shop/bigdiscounts/{orderId}",

My question is twofold:

1. Can I accomplish that only 10 requests per orderId are accepted. I think yes, use Period, Limit and PeriodTimespan.
2. Will this also work after an upgrade of Ocelot, or after the web server was restarted. I guess this question related directly to whether or not Ocelot keeps a database of incoming requests that it uses to determine how to act, like accept the request and route it to the appropiate downstream service, or response with a 429.

Thanks in advice.
Maurizio

[raman-m]

Maurizio, thanks for asking the question. What's your LinkedIn?

I don't see a forked repo un your GH repos. As far as I understood, you started to use Ocelot.
It would be worth seeing the complete configuration, aka ocelot.json file.

My aim is to limit the use of a specific route that contains a dynamic placeholder. For example consider "UpstreamPathTemplate": "/shop/bigdiscounts/{orderId}",

The Rate Limiting feature is agnostic to placeholders, since the rate limiting cache key (caching counter value for a route) includes the upstream path as part of the caching key.

Ocelot/src/Ocelot/RateLimiting/RateLimitingMiddleware.cs

Line 107 in e324bba

return new ClientRequestIdentity(clientId, route.LoadBalancerKey);

Here are the parts that the load balancing key consists of:

Ocelot/src/Ocelot/Configuration/Creator/RouteKeyCreator.cs

Lines 22 to 39 in e324bba

	public string Create(FileRoute route, LoadBalancerOptions loadBalancing)
	{
	if (TryStickySession(loadBalancing, out var stickySessionKey))
	{
	return stickySessionKey;
	}
	var keyBuilder = new StringBuilder()
	.AppendNext(route.UpstreamHttpMethod.Csv()) // required
	.AppendNext(route.UpstreamPathTemplate) // required
	.AppendNext(route.UpstreamHost.IfEmpty("no-host")) // optional...
	.AppendNext(route.DownstreamHostAndPorts.Select(AsString).Csv().IfEmpty("no-host-and-port"))
	.AppendNext(route.ServiceNamespace.IfEmpty("no-svc-ns"))
	.AppendNext(route.ServiceName.IfEmpty("no-svc-name"))
	.AppendNext(loadBalancing.Type.IfEmpty("no-lb-type"))
	.AppendNext(loadBalancing.Key.IfEmpty("no-lb-key"));
	return keyBuilder.ToString();
	}

Finally, each route has a unique load balancing key; thus, each route has a dedicated rate limiter.

---

Can I accomplish that only 10 requests per orderId are accepted. I think yes, use Period, Limit and PeriodTimespan.

If you have one route containing a {orderId} placeholder, then definitely you can. Just define RateLimitOptions.
If there are multiple routes containing a {orderId} placeholder, then you can rate limit each route by an independent limiter.

If you meant that the sum of all counters for all routes containing a {orderId} placeholder, then it is currently impossible. In this scenario, routes should be grouped somehow. Technically, you could override the following method of the RateLimitingMiddleware:

Ocelot/src/Ocelot/RateLimiting/RateLimitingMiddleware.cs

Line 98 in e324bba

protected virtual ClientRequestIdentity Identify(HttpContext context, RateLimitOptions options, DownstreamRoute route)

and the next step is injecting custom middleware into Ocelot pipeline here:

Ocelot/src/Ocelot/Middleware/OcelotPipelineExtensions.cs

Line 80 in e324bba

app.UseMiddleware<RateLimitingMiddleware>();

you see this middleware can't be overridden, but this restriction can be easily eliminated by patching the Ocelot pipeline configuration.
I can't provide you with a complete and workable solution since development takes time, so I'm letting you develop in your own forked repository (please fork the repo and develop).

---

Will this also work after an upgrade of Ocelot, or after the web server was restarted.

No, it won't. The rate limiter keeps counters in the cache, which is a memory cache by default and is added to the DI container:

Ocelot/src/Ocelot/DependencyInjection/Features.cs

Line 49 in e324bba

.AddSingleton<IRateLimitStorage, MemoryCacheRateLimitStorage>();

But there is good news, since there is a DistributedCacheRateLimitStorage class that can be added to DI. Thus, you can build a cluster of Ocelot instances, which will use the same rate limiting distributed storage (databases).

I guess this question related directly to whether or not Ocelot keeps a database of incoming requests that it uses to determine how to act, like accept the request and route it to the appropriate downstream service, or response with a 429.

Ocelot has no database, and it should not, due to Ocelot's major responsibility, which is routing and transforming traffic as a reverse proxy or as an API gateway. Tracking each request and keeping request data in a database will significantly degrade gateway performance.
Ocelot's Rate Limiting feature doesn't need any databases at all! It is designed to be abstract and depends on well-known Microsoft interfaces that provide caching or data storage.

If you want to attach a distributed database, you must do the following:

• Inject DistributedCacheRateLimitStorage into the DI container, replacing the default in-memory cache service.
• Embed or attach any database solution that relies on the .NET Framework's IDistributedCache interface. Thus, you have to inject caching software using this interface. Read the docs → Distributed caching in ASP.NET Core

After these manipulations, your rate limiting will support clustering (using shared distributed caches).
I hope this helps!

[raman-m]

@ms30063600 Your question regarding the database was already answered in 2024 (see pinned thread #2098):

• Does Ocelot support sharing routes and rate limit configurations between its instances if we deploy more than one Ocelot instance? #2098

[raman-m]

P.S.

As an option, if you want advanced and flexible rate limiters, you could utilize ASP.NET Core native middleware, aka Rate Limiter policies. We've described this in the RL Roadmap doc.

As far as I know, ASP.NET rate limiting can be global only, unfortunately. In theory, a policy could be applied to WebAPI endpoints or MVC controllers, but due to the fact that Ocelot's routing architecture is not based on endpoints or controllers, ASP.NET rate limiters are incompatible with Ocelot routes (PR #2188). Thus, you will be able to consume one global policy attached to all routes, which is absolutely useless for reverse proxying in API gateways having multiple routes. If there is one Catch All route then a global limiter will be useful. Even though it is global, it is extremely flexible — but this global scope is a restriction.

In comparison to the ASP.NET rate limiter, Ocelot's implementation provides an independent rate limiter for each route, which is extremely useful for API gateways. However, Ocelot's implementation has only a few limiter algorithms and doesn't provide telemetry data (pure event model). Teams opinion has been expressed in the Roadmap, and it seems other rate limiting partitions would be nice to have, such as rate limiting by client IP (feature request #1050). And, currently we only propose partitioning by client header (aka By API Key).

[ms30063600]

@ms30063600 Your question regarding the database was already answered in 2024 (see pinned thread #2098):

• Does Ocelot support sharing routes and rate limit configurations between its instances if we deploy more than one Ocelot instance? #2098

Hi @raman-m ,

I did not have the opportunity to check last week if someone already answered my question.
Thank you very much for answering my question. I really appreciate it.
You wrote that Ocelot's major responsibility is routing and transforming traffic as a reverse proxy or as an API gateway.
Indeed I reckon that the introduction of a database by Ocelot will significantly degrade gateway performance, so I understand this design decision. Also thank you very much for your pointers for alternatives.
Within my current project we use Ocelot since 2021, but it was only used as an api gateway. The rate limiting functionality was not used before.

Kind regards,
Maurizio.

[raman-m]

Within my current project we use Ocelot since 2021, but it was only used as an api gateway.

Good! About your company... What is co size, industry, website/country, and annual income/turnover?
Please also describe the project's technical stack and which Ocelot features were used.

The rate limiting functionality was not used before.

It is time to start using the rate limiting feature. As a maintainer, I would be interested in feedback about using this feature.
I will help you set it up correctly if you have any questions.

[raman-m]

and the next step is injecting custom middleware into Ocelot pipeline here:

Ocelot/src/Ocelot/Middleware/OcelotPipelineExtensions.cs

Line 80 in e324bba

app.UseMiddleware<RateLimitingMiddleware>();

you see this middleware can't be overridden, but this restriction can be easily eliminated by patching the Ocelot pipeline configuration.
I can't provide you with a complete and workable solution since development takes time, so I'm letting you develop in your own forked repository (please fork the repo and develop).

Regarding overriding the middleware... A week ago I merged interesting PR #2387, which provides the ability to override WebSocketsProxyMiddleware via a Type option in the OcelotPipelineConfiguration class. I have investigated the possible ways of middleware injection, and it seems specifying a Type in pipeline options is the simplest injection (the most intuitive user experience) for overriding Ocelot's default middlewares.

So, we could override RateLimitingMiddleware using the same approach. But we need to open a pull request...