Skip to content


Subversion checkout URL

You can clone with
Download ZIP


Add task queue limit in engine when load-balancing #399

kaazoo opened this Issue · 1 comment

2 participants


When load-balancing accross all engines, all newly created tasks are queued to the existing engines. When new engines arrive, they don't get any of these tasks, allthough lot's of them are waiting. Engines should have a configurable task queue limit, so the hub keeps the rest of the tasks and makes them available to other engines when they join the cluster.


Indeed, this is exactly what the HWM configurable is in the TaskScheduler, and merged just yesterday in commit e6ade7a. The default behavior remains unlimited, which has the lowest latency on a static engine set. You must use the Python scheduler (the default) for this to be available, as it is not possible with the pure-zmq scheduler.

@minrk minrk closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.