Polling strategies

Shoryuken has built in two polling strategies WeightedRoundRobin and StrictPriority, but you are free to extend it and use your own polling strategy in case you have a special need. In most cases, the default one (WeightedRoundRobin) is the way to go.

WeightedRoundRobin

WeightedRoundRobin is the default polling strategy, it works as follows:

Given the configuration below:

queues:
  - [queue1, 10]
  - [queue2, 5]
  - [queue3, 1]

When Shoryuken starts, all queues starts with priority 1.

Then Shoryuken starts fetching messages from queue1, if there are any message in the queue1, it increases the priority of queue1 by 1, until it reaches the max available weight 10. Same for queue2 and queue1.

Supposing all queues are full of messages, how it would look like:

1. fetches messages from queue1
2. fetches messages from queue2
3. fetches messages from queue3
4. fetches messages from queue1
5. fetches messages from queue1
6. fetches messages from queue2
7. fetches messages from queue2
8. fetches messages from queue3

The same incremental flow will continue until queue1 reaches 10, queue2 reaches 5 and queue3 will be kept at 1.

If delay is configured:

delay: 10
queues:
  - [queue1, 10]
  - [queue2, 5]
  - [queue3, 1]

And a queue gets empty, the queue will removed from the polling list for the number of seconds specified in delay. The default delay is 0. For example, if queue1 was with its max weight 10, and then it gets empty, it will be removed from the polling list, and in 10 seconds later it will be added back with priority 1.

StrictPriority

StrictPriority will cause Shoryuken to keep fetching messages from the highest priority queue until it returns no messages, it will then try the next highest priority queue, etc.

queues:
  - [queue1, 3]
  - [queue2, 2]
  - [queue3, 1]

While queue1 has messages, Shoryuken won't try to fetch message from queue2 or queue1.

delay is also supported for StrictPriority. With delay being 0, every iteration will cause Shoryuken to try the queues in priority order, stopping once it finds a job.

For switching to StrictPriority:

Shoryuken.configure_server do |config|
  config.options[:polling_strategy] = Polling::StrictPriority
end