Rate Limit proposal

[lobkovilya]

Proposal on the new Rate Limit policy (or maybe Rate Limiting?)

[subnetmarco]

max becomes max_burst_allowed and when not specified by default it always matches the connection (or requests).

[jakubdyszkiewicz]

@subnetmarco

What is the main use case that users and customers are looking for?

Is it about throttling web so it won't send more than 100 req/s to backend if app is misbehaving or is it about protecting backend that it does not receive more than X req/s?

If former, then we can go with Ilya's proposal, but if it is about protecting backend then it should be

type: RateLimit
name: rt-1
mesh: default
selectors:
- match:
    kuma.io/service: backend # apply on backend for any traffic
conf:
  tcp:
    connections: 100 # how many connections are allowed per 'period'
    period: 1s
  http: ...

To give more examples with the current approach

---
type: RateLimit
name: rt-1
mesh: default
sources:
- match:
    kuma.io/service: web
destinations:
- match:
    kuma.io/service: backend
conf:
  http:
    requests: 100 # how many requests are allowed per 'period'
    period: 1s
---
type: RateLimit
name: rt-1
mesh: default
sources:
- match:
    kuma.io/service: another-client
destinations:
- match:
    kuma.io/service: backend
conf:
  http:
    requests: 100 # how many requests are allowed per 'period'
    period: 1s

web AND another-client are able to execute 100 req/s.

type: RateLimit
name: rt-1
mesh: default
sources:
- match:
    kuma.io/service: '*'
destinations:
- match:
    kuma.io/service: backend
conf:
  http:
    requests: 100 # how many requests are allowed per 'period'
    period: 1s

any instance of a service can execute 100 req/s

[lobkovilya]

Why the approach with sources and destinations can't have a policy like this:

type: RateLimit
name: rt-1
mesh: default
sources:
- match:
    kuma.io/service: *
destinations:
- match:
    kuma.io/service: backend
conf:
  http:
    requests: 100 # how many requests are allowed per 'period'
    period: 1s

[jakubdyszkiewicz]

It can, but again. This is different than

type: RateLimit
name: rt-1
mesh: default
selectors:
- match:
    kuma.io/service: backend # apply on backend for any traffic
conf:
  tcp:
    connections: 100 # how many connections are allowed per 'period'
    period: 1s
  http: ...

applied on the backend.

In your case, any instance can send up to 100 req/s to backend.
In the case above, one instance of backend can process 100 req/s.

[lobkovilya]

but I proposed to apply rate-limiting on the destination side, selecting by sources might be tricky but I think it's possible to implement at least for HTTP

[lobkovilya]

@jakubdyszkiewicz @subnetmarco I reflected this discussion as 3 different options in the proposal

[subnetmarco]

Rate limiting is for protection of the destination. Although the destination service may want to assign different limits to different sources. Use cases are:

• backend will only handle 100 req/s from any source.
• backend will only handle 100 req/s from any source, but a specific backend-app source should be able to make 200req/s.

[jpeach]

So this is a local rate limit on each backend instance?

[lobkovilya]

Yes, that's correct

[subnetmarco]

This also needs to support rate limiting on external services, therefore in this case applied at the source and not at the destination.

[subnetmarco]

Also, we need to support multiple backends for rate limiting. In this case is local but we may want to store redis in the future, like:

type: RateLimit
name: rt-1
mesh: default
sources:
- match:
    kuma.io/service: edge-gateway
destinations:
- match:
    kuma.io/service: httpbin
conf:
  backends:
    local: {}
    redis:
      address: 1.1.1.1
      port: 8888
  http:
    requests: 100
    period: 1s
    status: 401

[nickolaev]

This also needs to support rate limiting on external services, therefore in this case applied at the source and not at the destination.

The implementation that I have been working on in #2083 is done on the destination.

[jpeach]

This also needs to support rate limiting on external services, therefore in this case applied at the source and not at the destination.

The implementation that I have been working on in #2083 is done on the destination.

IMHO the different places rate limits can be applied are useful for different operational reasons. Since it's likely that we will end up with different kinds of rate limit (e.g global, egress, ingress), we just need to be clear about it in the API design and documentation.

[jpeach]

Is this done since #2083 is merged?