Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TaskScheduler.hwm default value #1293

Closed
minrk opened this issue Jan 19, 2012 · 2 comments · Fixed by #1294
Closed

TaskScheduler.hwm default value #1293

minrk opened this issue Jan 19, 2012 · 2 comments · Fixed by #1294
Labels
parallel
Milestone
0.13

Comments

@minrk
Copy link
Member

minrk commented Jan 19, 2012

The current default value is 0, which minimizes task latency with greedy assignment. However, this is suboptimal for some (especially heterogeneous) workloads, and surprising for many users. It may be more sensible to make the default value 1, which is more friendly for novice users, who can get confused by the greedy assignment.

This is a tiny change, but posting here so there's a record of the decision.
@fperez
Copy link
Member

fperez commented Jan 19, 2012

+1 for the change, thanks for making the record here!

@minrk minrk mentioned this issue Jan 20, 2012
Merged
@minrk
Copy link
Member Author

minrk commented Jan 20, 2012

opened PR #1294.

Part of the case for keeping 0:

  • reduced memory load on the Scheduler. When hwm=1 the entire queue resides there instead of being distributed across engines
  • most closely resembles pure-zmq scheduler

@fperez fperez closed this as completed in 0cde7de Jan 20, 2012
mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this issue Nov 3, 2014
@fperez
Merge pull request ipython#1294 from minrk/hwm1
574d01b 
TaskScheduler.hwm default to 1 instead of 0.  1 has more predictable/intuitive behavior, if often slower, and thus a more logical default.  Users whose workloads require maximum throughput and are largely homogeneous in time per task can make the optimization themselves, but now the behavior will be less surprising to new users.

closes ipython#1293
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
parallel
Projects
None yet
Milestone
0.13
Development

Successfully merging a pull request may close this issue.

TaskScheduler.hwm default to 1 instead of 0 minrk/ipython
2 participants
@fperez @minrk