Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report currentActiveInstances on SingularityDeployProgress #1377

Merged
merged 6 commits into from
Feb 15, 2017

Conversation

matush-v
Copy link
Contributor

@matush-v matush-v commented Dec 15, 2016

This reports that a canary deploy has completed the first step

@matush-v matush-v changed the title set canary deploy started status after first step of incremental deploy Canary deploy Dec 15, 2016
"Pending deploy already in progress for %s - cancel it or wait for it to complete (%s)", requestId, deployManager.getPendingDeploy(requestId).orNull());
boolean deployExists = deployManager.createPendingDeploy(pendingDeployObj) == SingularityCreateResult.EXISTED;
if (deployExists && forceDeploy.or(false)) {
cancelDeploy(requestId, deployManager.getInUseDeployId(requestId).orNull());
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unsure if there is a restart method or if I need to do something after hitting the cancel

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we implement force deploy in a separate PR?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah agreed on leaving force deploy for a separate PR, we'd need to cancel + make sure the deploy checker successfully cancelled before starting a new one for things to work properly.

}

public SingularityDeployProgress withCompletedStep() {
return new SingularityDeployProgress(targetActiveInstances, deployInstanceCountPerStep, deployStepWaitTimeMs, true, autoAdvanceDeploySteps, failedDeployTasks, System.currentTimeMillis());
return new SingularityDeployProgress(targetActiveInstances, currentActiveInstances + 1, deployInstanceCountPerStep, deployStepWaitTimeMs, true, autoAdvanceDeploySteps, failedDeployTasks, System.currentTimeMillis());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is currentActiveInstances + 1 correct here? wouldn't it be currentActiveInstances + deployInstanceCountPerStep?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was originally intended to just be the current deploy progress with stepCompleted flipped to true. If we want to update the active instance count I'd suggest getting it from the deployActiveTasks variable that gets passed around instead. If we pass that to the calls of markStepFinished, we can construct the new deploy progress with the actual count of active tasks after each step. withCompletedStep should do just what I mentioned above and flip the value of that bool, keeping the rest the same

"Pending deploy already in progress for %s - cancel it or wait for it to complete (%s)", requestId, deployManager.getPendingDeploy(requestId).orNull());
boolean deployExists = deployManager.createPendingDeploy(pendingDeployObj) == SingularityCreateResult.EXISTED;
if (deployExists && forceDeploy.or(false)) {
cancelDeploy(requestId, deployManager.getInUseDeployId(requestId).orNull());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we implement force deploy in a separate PR?

@matush-v
Copy link
Contributor Author

@tpetr any changes necessary on this one?

@tpetr
Copy link
Contributor

tpetr commented Dec 27, 2016

Looks good but could you add unit tests for this? Or if there are already tests flexing this functionality, update them to ensure that currentActiveInstances is updated properly?

@matush-v
Copy link
Contributor Author

Just added a test following the logic of existing tests, but I can't say I'm super confident that it's the best way to test the counter.

e.g. I couldn't figure out how to deploy with a target of 4 instances, and check the currentActiveInstances one at a time. I used the logic in another test that seemed to update the request each time before checking.

Thoughts? bitmoji

@ssalinas
Copy link
Member

@darcatron apologies for letting this one sit so long. I think having that current active instances value will still be valuable. In terms of the test, I would check out the example of testDeployOneInstanceAtATime in the same test class for how to do one step at a time of an incremental deploy. What that ones does is it:

  • launches two initial tasks for wht will be the 'old' deploy
  • starts a new deploy with incremental settings (1 instance at a time)
  • updates the first task to be running and runs the deploy check to update the deploy progress (which would now have a current active of 1)
  • makes sure the first old task gets killed
  • does the same for the second task

You could likely extend that test to do three instances instead of just two and add a few more assert statements in there to adequately test the new counter

@ssalinas ssalinas changed the title Canary deploy Report currentActiveInstances on SingularityDeployProgress Feb 2, 2017
@matush-v
Copy link
Contributor Author

matush-v commented Feb 2, 2017

@ssalinas gave the test another look based on your suggestions and it seems that was the original test I followed. Instead of adding a check for these changes by extending an existing test, this one is separate. I think that's probably better anyway so unless you want to combine them, I'm going to keep it as is.

Cool if I get this into staging?

@ssalinas
Copy link
Member

ssalinas commented Feb 2, 2017

I think that's fair, we can try this out in staging

@ssalinas ssalinas modified the milestone: 0.14.0 Feb 14, 2017
@ssalinas ssalinas merged commit f1ea1c5 into master Feb 15, 2017
@ssalinas ssalinas deleted the bird-deploy branch February 15, 2017 14:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants