Report currentActiveInstances on SingularityDeployProgress #1377

Merged
merged 6 commits into from Feb 15, 2017

Conversation

Projects
None yet
3 participants
@darcatron
Contributor

darcatron commented Dec 15, 2016

This reports that a canary deploy has completed the first step

@darcatron darcatron changed the title from set canary deploy started status after first step of incremental deploy to Canary deploy Dec 15, 2016

- "Pending deploy already in progress for %s - cancel it or wait for it to complete (%s)", requestId, deployManager.getPendingDeploy(requestId).orNull());
+ boolean deployExists = deployManager.createPendingDeploy(pendingDeployObj) == SingularityCreateResult.EXISTED;
+ if (deployExists && forceDeploy.or(false)) {
+ cancelDeploy(requestId, deployManager.getInUseDeployId(requestId).orNull());

This comment has been minimized.

@darcatron

darcatron Dec 15, 2016

Contributor

Unsure if there is a restart method or if I need to do something after hitting the cancel

@darcatron

darcatron Dec 15, 2016

Contributor

Unsure if there is a restart method or if I need to do something after hitting the cancel

This comment has been minimized.

@tpetr

tpetr Dec 15, 2016

Member

Can we implement force deploy in a separate PR?

@tpetr

tpetr Dec 15, 2016

Member

Can we implement force deploy in a separate PR?

This comment has been minimized.

@ssalinas

ssalinas Dec 15, 2016

Member

Yeah agreed on leaving force deploy for a separate PR, we'd need to cancel + make sure the deploy checker successfully cancelled before starting a new one for things to work properly.

@ssalinas

ssalinas Dec 15, 2016

Member

Yeah agreed on leaving force deploy for a separate PR, we'd need to cancel + make sure the deploy checker successfully cancelled before starting a new one for things to work properly.

}
public SingularityDeployProgress withCompletedStep() {
- return new SingularityDeployProgress(targetActiveInstances, deployInstanceCountPerStep, deployStepWaitTimeMs, true, autoAdvanceDeploySteps, failedDeployTasks, System.currentTimeMillis());
+ return new SingularityDeployProgress(targetActiveInstances, currentActiveInstances + 1, deployInstanceCountPerStep, deployStepWaitTimeMs, true, autoAdvanceDeploySteps, failedDeployTasks, System.currentTimeMillis());

This comment has been minimized.

@tpetr

tpetr Dec 15, 2016

Member

is currentActiveInstances + 1 correct here? wouldn't it be currentActiveInstances + deployInstanceCountPerStep?

@tpetr

tpetr Dec 15, 2016

Member

is currentActiveInstances + 1 correct here? wouldn't it be currentActiveInstances + deployInstanceCountPerStep?

This comment has been minimized.

@ssalinas

ssalinas Dec 15, 2016

Member

This was originally intended to just be the current deploy progress with stepCompleted flipped to true. If we want to update the active instance count I'd suggest getting it from the deployActiveTasks variable that gets passed around instead. If we pass that to the calls of markStepFinished, we can construct the new deploy progress with the actual count of active tasks after each step. withCompletedStep should do just what I mentioned above and flip the value of that bool, keeping the rest the same

@ssalinas

ssalinas Dec 15, 2016

Member

This was originally intended to just be the current deploy progress with stepCompleted flipped to true. If we want to update the active instance count I'd suggest getting it from the deployActiveTasks variable that gets passed around instead. If we pass that to the calls of markStepFinished, we can construct the new deploy progress with the actual count of active tasks after each step. withCompletedStep should do just what I mentioned above and flip the value of that bool, keeping the rest the same

- "Pending deploy already in progress for %s - cancel it or wait for it to complete (%s)", requestId, deployManager.getPendingDeploy(requestId).orNull());
+ boolean deployExists = deployManager.createPendingDeploy(pendingDeployObj) == SingularityCreateResult.EXISTED;
+ if (deployExists && forceDeploy.or(false)) {
+ cancelDeploy(requestId, deployManager.getInUseDeployId(requestId).orNull());

This comment has been minimized.

@tpetr

tpetr Dec 15, 2016

Member

Can we implement force deploy in a separate PR?

@tpetr

tpetr Dec 15, 2016

Member

Can we implement force deploy in a separate PR?

@darcatron

This comment has been minimized.

Show comment
Hide comment
@darcatron

darcatron Dec 27, 2016

Contributor

@tpetr any changes necessary on this one?

Contributor

darcatron commented Dec 27, 2016

@tpetr any changes necessary on this one?

@tpetr

This comment has been minimized.

Show comment
Hide comment
@tpetr

tpetr Dec 27, 2016

Member

Looks good but could you add unit tests for this? Or if there are already tests flexing this functionality, update them to ensure that currentActiveInstances is updated properly?

Member

tpetr commented Dec 27, 2016

Looks good but could you add unit tests for this? Or if there are already tests flexing this functionality, update them to ensure that currentActiveInstances is updated properly?

@darcatron

This comment has been minimized.

Show comment
Hide comment
@darcatron

darcatron Dec 28, 2016

Contributor

Just added a test following the logic of existing tests, but I can't say I'm super confident that it's the best way to test the counter.

e.g. I couldn't figure out how to deploy with a target of 4 instances, and check the currentActiveInstances one at a time. I used the logic in another test that seemed to update the request each time before checking.

Thoughts? bitmoji

Contributor

darcatron commented Dec 28, 2016

Just added a test following the logic of existing tests, but I can't say I'm super confident that it's the best way to test the counter.

e.g. I couldn't figure out how to deploy with a target of 4 instances, and check the currentActiveInstances one at a time. I used the logic in another test that seemed to update the request each time before checking.

Thoughts? bitmoji

@ssalinas

This comment has been minimized.

Show comment
Hide comment
@ssalinas

ssalinas Jan 23, 2017

Member

@darcatron apologies for letting this one sit so long. I think having that current active instances value will still be valuable. In terms of the test, I would check out the example of testDeployOneInstanceAtATime in the same test class for how to do one step at a time of an incremental deploy. What that ones does is it:

  • launches two initial tasks for wht will be the 'old' deploy
  • starts a new deploy with incremental settings (1 instance at a time)
  • updates the first task to be running and runs the deploy check to update the deploy progress (which would now have a current active of 1)
  • makes sure the first old task gets killed
  • does the same for the second task

You could likely extend that test to do three instances instead of just two and add a few more assert statements in there to adequately test the new counter

Member

ssalinas commented Jan 23, 2017

@darcatron apologies for letting this one sit so long. I think having that current active instances value will still be valuable. In terms of the test, I would check out the example of testDeployOneInstanceAtATime in the same test class for how to do one step at a time of an incremental deploy. What that ones does is it:

  • launches two initial tasks for wht will be the 'old' deploy
  • starts a new deploy with incremental settings (1 instance at a time)
  • updates the first task to be running and runs the deploy check to update the deploy progress (which would now have a current active of 1)
  • makes sure the first old task gets killed
  • does the same for the second task

You could likely extend that test to do three instances instead of just two and add a few more assert statements in there to adequately test the new counter

@ssalinas ssalinas changed the title from Canary deploy to Report currentActiveInstances on SingularityDeployProgress Feb 2, 2017

@darcatron

This comment has been minimized.

Show comment
Hide comment
@darcatron

darcatron Feb 2, 2017

Contributor

@ssalinas gave the test another look based on your suggestions and it seems that was the original test I followed. Instead of adding a check for these changes by extending an existing test, this one is separate. I think that's probably better anyway so unless you want to combine them, I'm going to keep it as is.

Cool if I get this into staging?

Contributor

darcatron commented Feb 2, 2017

@ssalinas gave the test another look based on your suggestions and it seems that was the original test I followed. Instead of adding a check for these changes by extending an existing test, this one is separate. I think that's probably better anyway so unless you want to combine them, I'm going to keep it as is.

Cool if I get this into staging?

@ssalinas

This comment has been minimized.

Show comment
Hide comment
@ssalinas

ssalinas Feb 2, 2017

Member

I think that's fair, we can try this out in staging

Member

ssalinas commented Feb 2, 2017

I think that's fair, we can try this out in staging

@ssalinas ssalinas added the hs_stable label Feb 13, 2017

@ssalinas ssalinas modified the milestone: 0.14.0 Feb 14, 2017

@ssalinas ssalinas merged commit f1ea1c5 into master Feb 15, 2017

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details

@ssalinas ssalinas deleted the bird-deploy branch Feb 15, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment