This code aims to help setting up a queue management with specific needs:
- multitenant application
- numner of worker per tenant is limited
- 1 worker / job / time
- sequential jobs
- dynamic queues ? depend on how many consumer
- on consumer will send job1 and job2 in sequence for 100 messages.
- job1 and job2 cannot operate at same time: latency, timeout
- queue 100 messages for job1
- result of job1 is queued for job2
-
- share workers among queues
Looking at RabbitMQ, ZeroMQ, Sidekiq
- set of waiting queues
- list of waiting queue
- pop a queue to start a job
- remove from set when job ends
- if message wait in queue then add to set
- available queues: set
- waiting queues: set + list
- working queues: set