Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce an incremental bounce #767

Merged
merged 13 commits into from Dec 10, 2015
Merged

Introduce an incremental bounce #767

merged 13 commits into from Dec 10, 2015

Conversation

ssalinas
Copy link
Member

Currently, in order to bounce a request which has SEPARATE* placement, you need to be running instance_count * 2 slaves. This can be quite a pain for requests with many instances. This PR creates a new type of bounce where we shut down old tasks as new ones are spun up (vs waiting for all new tasks to be ready first). Using this, you only need a minimum of instance_count + 1 slaves to bounce the same request

@ssalinas ssalinas changed the title (WIP) introduce an incremental bounce Introduce an incremental bounce Nov 24, 2015
@ssalinas
Copy link
Member Author

Got this working well in staging, also added an option for it in the UI
screen shot 2015-11-24 at 4 48 01 pm

@@ -362,9 +386,19 @@ private void drainTaskCleanupQueue() {
return;
}

Map<String, Integer> incrementalBounceRemainingInstanceMap = new HashMap<>();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you clarify what this map does? it tracks how many tasks still need to bounce per request?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When doing our cleanup, each task doesn't know what has happened to the task before it. This way we can keep track of how many cleaning tasks were already killed in this cleanup run. (i.e. if we have room to kill one cleaning task in our incremental bounce, now we'll know when we get to the second that we've already done that and we shouldn't kill the second)

@ssalinas ssalinas added the hs_qa label Dec 7, 2015
@tpetr
Copy link
Contributor

tpetr commented Dec 9, 2015

LGTM

@tpetr tpetr added this to the 0.4.6 milestone Dec 9, 2015
ssalinas added a commit that referenced this pull request Dec 10, 2015
@ssalinas ssalinas merged commit bfbeec3 into master Dec 10, 2015
@ssalinas ssalinas deleted the incremental_bounce branch December 10, 2015 14:36
@ssalinas ssalinas mentioned this pull request Dec 14, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants