"Fair" dequeuing across customers and customerio Fairway

[woodhull]

Hello!

I'm a very happy sidekiq user across two different companies.

At one of my businesses, a multi-tennant SaaS App we have a downstream service-provider that limits our overall capacity. They can only process a certain number of API calls per second. We use rate-limiting for this and it works well.

When multiple customers have messages to send, our current sidekiq queues mean that the first customer to enqueue their jobs "wins" and the other customers must wait for jobs to be processed.

It's instead more desirable for API calls to be interleaved between customer accounts so everyone gets a fair share of the capacity.

This is the same use-case as Customer.io worked on in this project: https://github.com/customerio/fairway

However, that code base does not seem to have recent activity, and looks as if it might not be maintained. Are there any alternative architectures that might make sense here? The other idea we had is that we could have 10 queues and when work needs to be done, assign jobs randomly to them. Sidekiq workers would then read across all of these queues.

Are there any other recommended architectures or patterns for this sort of situation? From what I understand of how sidekiq job fetching works it would be bad to have a queue per-customer (we have potentially 100s, most of them would be empty almost all of the time).

This is particularly tricky for our customer base which tends to be most active on the same small number of days each year. On these days we have very high levels of contention for this shared & rate-limited third party API resource.

Welcome ideas or pointers to other projects like the one from customer.io.

Nathan

[woodhull]

Our other idea is to modernize and revive the code from customer.io. if there is not another obvious path forward.

[mperham]

https://www.mikeperham.com/2019/12/17/workload-isolation-with-queue-sharding/

[woodhull]

Thanks, this is a very good architecture and exactly what we needed. Maybe there is some way to include a link to this blog post in the Sidekiq wiki so its more discoverable while reading the docs. Perhaps in the "Best Practices" section?

https://github.com/sidekiq/sidekiq/wiki/Best-Practices

[mperham]

It's an extremely niche and advanced topic, requiring good devops savvy. The wiki is meant for all, the Best Practices page is meant for beginners, Advanced Options is meant for more technical options which are still broadly relevant. I think I will put it on the wiki home page, at the bottom.

[woodhull]

Yeah just thinking about how to improve & eliminate the question from the next person who comes along.