Skip to content

Spread to all slaves - #1519

Merged
ssalinas merged 4 commits into
masterfrom
spread_to_all_slaves
May 1, 2017
Merged

ssalinas merged 4 commits into
masterfrom
spread_to_all_slaves

Conversation

@ssalinas

Copy link
Copy Markdown
Contributor

@felixgborrego I know this is long overdue to get this out there. Opening this PR up so I could make a few tweaks and fix some merge conflicts with our recent changes. A few things:

  • I'm still not wild that the solution has no concrete guarantee that the tasks will get put on all slaves. That said, the alternatives like hooking up support for resource reservations are a much larger undertaking.
  • We recently had some performance issue around the runtime of pollers contending for the scheduler lock, so I was hesitant to add one more poller there.

However, since the feature is very opt-in, I added a simple configuration flag to turn the poller and feature on or off (the validator will throw you an error if you try to use the request type when it is not enabled). With this in place I think I am comfortable to merge it in for now. You will just need to add spreadAllSlavesEnabled: true to your singularity yaml when starting.

@ssalinas

Copy link
Copy Markdown
Contributor Author

Additional FYI @felixgborrego , merging this in our branches to give it one last test now and I will plan to do a release of 0.15.0 monday/tuesday next week.

@ssalinas ssalinas modified the milestone: 0.15.0 Apr 28, 2017
@relistan

Copy link
Copy Markdown
Contributor

Thanks @ssalinas, this is going to be great to have upstream!

@ssalinas
ssalinas merged commit 04ed331 into master May 1, 2017
@ssalinas
ssalinas deleted the spread_to_all_slaves branch May 1, 2017 13:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants