New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spread to all slaves #1519

Merged
merged 4 commits into from May 1, 2017

Conversation

Projects
None yet
3 participants
@ssalinas
Member

ssalinas commented Apr 28, 2017

@felixgborrego I know this is long overdue to get this out there. Opening this PR up so I could make a few tweaks and fix some merge conflicts with our recent changes. A few things:

  • I'm still not wild that the solution has no concrete guarantee that the tasks will get put on all slaves. That said, the alternatives like hooking up support for resource reservations are a much larger undertaking.
  • We recently had some performance issue around the runtime of pollers contending for the scheduler lock, so I was hesitant to add one more poller there.

However, since the feature is very opt-in, I added a simple configuration flag to turn the poller and feature on or off (the validator will throw you an error if you try to use the request type when it is not enabled). With this in place I think I am comfortable to merge it in for now. You will just need to add spreadAllSlavesEnabled: true to your singularity yaml when starting.

@ssalinas

This comment has been minimized.

Show comment
Hide comment
@ssalinas

ssalinas Apr 28, 2017

Member

Additional FYI @felixgborrego , merging this in our branches to give it one last test now and I will plan to do a release of 0.15.0 monday/tuesday next week.

Member

ssalinas commented Apr 28, 2017

Additional FYI @felixgborrego , merging this in our branches to give it one last test now and I will plan to do a release of 0.15.0 monday/tuesday next week.

@ssalinas ssalinas modified the milestone: 0.15.0 Apr 28, 2017

@relistan

This comment has been minimized.

Show comment
Hide comment
@relistan

relistan Apr 28, 2017

Thanks @ssalinas, this is going to be great to have upstream!

relistan commented Apr 28, 2017

Thanks @ssalinas, this is going to be great to have upstream!

@ssalinas ssalinas merged commit 04ed331 into master May 1, 2017

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details

@ssalinas ssalinas deleted the spread_to_all_slaves branch May 1, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment