Limit the number of sidekiq queue executions

[kaku0225]

Hello, I am kaku

In my case, the high-priority queue is stuck because it takes too long to execute and affects the execution of other queues. Then it all got stuck.

Is there a way to do this in sidekiq pro or sidekiq enterprise?　↓

concurrency: 10
important takes 5
default takes 5

Even important is stuck, there are still 5, which can be used by default

I have found the gem sidekiq-limit_fetch, which can meet my needs
But if it is updated later, it may bring instability, so I hope it is the function provided by sidekiq

Could you please help me?

thanks

[nateberkopec]

Weighted queues would accomplish what you want:

---
:queues:
  - [high, 1]
  - [default, 1]
:concurrency: 10

This configuration means that the next job that is executed has a 50% chance of being from the high queue and a 50% chance of being from the default queue.

You could also make Capsules when Sidekiq 7 comes out. Using your example:

Sidekiq.configure_server do |config|
  config.capsule("important") do |cap|
    cap.concurrency = 5
    cap.queues = %w[important]
  end
  config.capsule("default") do |cap|
    cap.concurrency = 5
    cap.queues = %w[default]
  end
end

This would have an advantage over weighted queues - the concurrency of each queue is limited to 5. With weighted queues, it is random, so you could end up 10 workers executing from one queue by random chance.