Ent Rate Limiting

Often 3rd party APIs will enforce a rate limit, meaning you cannot call them faster than your SLA allows. Sidekiq Enterprise contains a rate limiting API with three styles of rate limiting: concurrent, window and bucket. This feature requires Redis 2.8+.

Note: limiters are somewhat heavyweight to create, requiring a round-trip to Redis. If possible, create limiters once during startup and reuse them (as with ERP_THROTTLE below). They are thread-safe and designed to be shared.

Concurrent

The concurrent style means that only N concurrent operations can happen at any moment in time. For instance, I've used an ERP SaaS which limited each customer to 50 concurrent operations. Use a concurrent rate limiter to ensure your processes stay within that rate limit:

ERP_THROTTLE = Sidekiq::Limiter.concurrent('erp', 50, wait_timeout: 5, lock_timeout: 30)

def perform(...)
  ERP_THROTTLE.within_limit do
    # call ERP
  end
end

Since concurrent access has to hold a lock, the lock_timeout option ensures a crashed Ruby process does not hold a lock forever. You must ensure that your operations take less than this number of seconds. After lock_timeout seconds, the lock can be reclaimed by another thread wanting to perform an operation.

You can use a concurrent limiter of size 1 to make a distributed mutex, ensuring that only one process can execute a block at a time.

Concurrent limiters will pause up to wait_timeout seconds for a lock to become available. This API is blocking and as efficient as possible: it does not poll unlike most other locking or mutex libraries for Redis. Blocking ensures the lock will be made available to a waiter within milliseconds of it being released.

Bucket

Bucket means that each interval is a bucket: you can perform 5 operations at 12:42:51.999 and then another 5 operations at 12:42:52.000 because they are tracked in a different bucket.

Here's an example using a bucket limiter of 30 per minute (notice how the name includes the user's ID, making it a user-specific limiter). Let's say we want to call Stripe on behalf of a user:

def perform(user_id)
  user_throttle = Sidekiq::Limiter.bucket("stripe-#{user_id}", 30, :minute, wait_timeout: 5)
  user_throttle.within_limit do
    # call stripe with user's account creds
  end
end

The limiter will try to perform the operation once per second until wait_timeout is passed or the rate limit is satisfied. It calls sleep to achieve this so the worker thread is paused during that sleep time.

You can also use :minute, :hour or :day buckets but they will not sleep until the next interval and retry the operation. They immediately raise Sidekiq::Limiter::OverLimit.

Window

Window means that each interval is a sliding window: you can perform N operations at 12:42:51.999 but can't perform another N operations until 12:42:52.999.

Here's an example using a window limiter of 5 per second (notice how the name includes the user's ID, making it a user-specific limiter). Let's say we want to call Stripe on behalf of a user:

def perform(user_id)
  user_throttle = Sidekiq::Limiter.window("stripe-#{user_id}", 5, :second, wait_timeout: 5)
  user_throttle.within_limit do
    # call stripe with user's account creds
  end
end

A :second limiter will try to perform the operation every half second until wait_timeout is passed or the rate limit is satisfied. It calls sleep to achieve this so the worker thread is paused during that sleep time.

You can also use :minute, :hour or :day buckets but they will not sleep until the next interval and retry the operation. They immediately raise Sidekiq::Limiter::OverLimit.

Take it to the Limit

If the rate limit is breached and cannot be satisfied within wait_timeout, the Limiter will raise Sidekiq::Limiter::OverLimit. If the limiter is being used within a Sidekiq worker, Sidekiq will rescue this error and reschedule the job to run at a random time in the future so as to avoid a "thundering herd" of jobs.

2015-05-28T23:25:23.159Z 73456 TID-oxf94yioo LimitedWorker JID-41c51a2123eef30dbad4544a INFO: erp over rate limit, rescheduling for later

Redis

Rate limiting is shared by ALL processes using the same Redis instance. If you have 50 Ruby processes connected to a Redis instance, they will all use the same rate limits. You can configure the Redis instance used by the rate limiting by setting a custom connection pool:

Sidekiq::Limiter.redis = ConnectionPool.new(size: 10) { Redis.new }

By default, the Sidekiq::Limiter API uses Sidekiq's default Redis pool so you don't need to configure anything.

Web UI

The Web UI contains a "Limits" tab which lists all limits configured in the system. Concurrent limiters track a number of metrics and expose those metrics in the UI.

Bucket limiters track recent history so you can see a graph of recent usage.

Note the one spike of 5, meaning the rate limit had to kick in. The graph shows "attempts" so the peak of 5 means one thread paused until the next second and retried the operation at that time.