Rate Limiting

[suresh-shetiar]

API rate limiting as an added security feature.

[adnankhan-neo]

@suresh-shetiar this would be done at API gateway level or at code base?

[suresh-shetiar]

Depends, because what if we are only running it under a server not using any API gateway. At that time we would need it to be in code base, and if we are going to use the API gateway (which might be not possible in all case) then a config on gateway level will work.

This could be also used to limit the API request based on server size to help prevent API from throttling.
Hence to be used in code base.

link to rate limiting example

[adnankhan-neo]

Okay @suresh-shetiar this sounds good. In order to improve performance for our node apps we can leverage usage of PM2 cluster mode also which allows us to scale the app across available vCPUs. This could be added as a separate discussion altogether also I feel

[PrajwalKarale08]

We are plaaning to proceed with NestJS/ Throttle package.

[PrajwalKarale08]

What is Rate Limiting/ API Throttling?

• Rate limiting is a technique that controls the consumption of resources used by an instance of an application.

Why Rate Limiting/ API Throttling?

• Protection against DDOS.

• Server Stability and Consistency.

• Cost Control

• Users can only do as much as is allowed by the developer.

There must be a clearly defined constraint for implementing rate limiting which could be as follows:
Users.

• Here the constraint is specific to a user and is implemented using a unique user identifier

Location.

• Here the constraint is based on geography and is implemented based on the location from which the request was made

IP Address.

• Here the constraint is based on the IP address of the device that initiates a request.

Personally I feel IP Address is the proper metric that can be used in order to keep an eye on the requests that is made since Location needs to be more precise which seems to be a tedious task.

How do Applications Rate Limit/Techniques for rate limitations.

Fixed Window Counter.

Window in this context refers to the space of time under consideration. That is, if I want my API to allow 10 requests per minute, we have a 60-second window. So, starting at 00:00:00, one window will be 00:00:00 to 00:01:00.

When user makes a first request within the given time limit then it is stored using a optimized key-value store(eg. redis). On subsequent request from the same window we have to whether the user have exceeded the limit or not. If it hasn’t then increase the count by one else drop the request and trigger the error msg.

Pros: This approach is relatively easy to implement.

Cons: In especially larger window cycles — e.g., 50 requests per hour (3,600 seconds) — the user may end up waiting for a very long time if they reach the limit in the first 10 minutes (600 seconds). That means it takes the user 10 minutes to make 50 requests, but one hour to make 51. This may result in a stampeding of the API immediately after a new window opens up.

Sliding Logs

The sliding logs algorithm keeps track of the timestamp for each request a user makes

The process of logging the requests is illustrated below:

Retrieve all requests logged in the last window (60 seconds) and check if the number of requests exceeds the allowed limit

If the number of requests is less than the limit, log the request and process it

If the number of requests is equal to the limit, drop the request.

Pros: This approach is more accurate as it calculates the last window per user based on the user’s activity and does not impose a fixed window for all users

Cons: it is not memory-efficient because we end up storing a new entry for every request made.

Sliding Window Counter

This approach attempts to optimize some of the inefficiencies of both the fixed window counter and sliding logs technique. In this technique, the user’s requests are grouped by timestamp, and rather than log each request, we keep a counter for each group.

It keeps track of each user’s request count while grouping them by fixed time windows(usually a fraction of the limit’s window size).

Pros: This approach saves more memory because instead of creating a new entry for every request, we group requests by timestamp and increment the counter.

Token Bucket

In the token bucket algorithm, we simply keep a counter indicating how many tokens a user has left and a timestamp showing when it was last updated. This concept originates from packet-switched computer networks and telecommunication networks in which there is a fixed-capacity bucket to hold tokens that are added at a fixed rate (window interval).

Pros: This is an accurate approach as the window is not fixed across users and, as such, is determined based on a user’s activity. Memory consumption is minimal since you only have one entry per user, which is used to manage their activity (timestamp and available tokens) over time.

In most of the articles they have suggested to use Token Bucket algorithm or Sliding Window Counter Technique.

Rate Limiting Implications

• Server Perspective.
  Decide an identity for throttling application I.e IP Address, Client Name, API Key.
  Determine the applications volume breaking point.