Behavior of perform_in with weighted queues?

[key88sf]

Hi - I'm seeing a (strange?) behavior but not sure if it's expected or not.

I have three queues in my sidekiq process that are weighted (via sidekiq.yml):

:queues:
  - [high, 3]
  - [default, 2]
  - [low, 1]

I have a bunch of jobs enqueued in the default queue, and I also have a process which is calling LowQueueJob.perform_in(...) , and this is adding jobs to be processed on the low queue.

What I'm seeing in the Sidekiq UI is that all the threads are constantly processing ONLY the low queue jobs, even though it is weighted the least. So my question is -- is this because when using perform_in(), it will force sidekiq to process the job when the delay time has gone by, regardless of the queue weighting?

[mperham]

The scheduler merely pushes the scheduled job onto the queue at the scheduled time. A Sidekiq process still has to fetch it to execute and that fetch process is what is weighted.

Once a Low job is running, it is taking a Processor thread. If your High and Default jobs take 1 second and your Low job takes 10 minutes to execute, all of your Threads and the Busy page will be dominated by Low jobs. You can avoid this by using Capsules or spin up another Sidekiq process which does not process the low queue.

[key88sf]

Ok cool, thank you for the clarification. 👍