Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Add task queue limit in engine when load-balancing #399

Closed
kaazoo opened this Issue · 1 comment

2 participants

@kaazoo

When load-balancing accross all engines, all newly created tasks are queued to the existing engines. When new engines arrive, they don't get any of these tasks, allthough lot's of them are waiting. Engines should have a configurable task queue limit, so the hub keeps the rest of the tasks and makes them available to other engines when they join the cluster.

@minrk
Owner

Indeed, this is exactly what the HWM configurable is in the TaskScheduler, and merged just yesterday in commit e6ade7a. The default behavior remains unlimited, which has the lowest latency on a static engine set. You must use the Python scheduler (the default) for this to be available, as it is not possible with the pure-zmq scheduler.

@minrk minrk closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.