Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(rollback): Ensure that rollback stages are injected correctly #2072

Merged
merged 2 commits into from
Mar 22, 2018

Conversation

ajordens
Copy link
Contributor

This PR handles generating a refId for a synthetic stage that has
pre-existing siblings.

In such cases, the refId should be unique and offset by the number of
siblings.

All onFailureStages() should be run linearly otherwise you run the
risk of a parent stage being notified prematurely.

@ajordens
Copy link
Contributor Author

image

Copy link
Contributor Author

@ajordens ajordens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to update the existing regression test you added.

Specifically looking to reproduce the scenario where
(a) there are stages that previously SUCCEEDED and (b) more than one on failure stage.

(a) will validate that we're not duplicating a refId.
(b) will validate that we're correctly connecting stages.

@@ -106,7 +106,6 @@ class CreateServerGroupStage extends AbstractDeployStrategyStage {
}

if (strategySupportsRollback) {
// TODO: should this append after any existing stages in the graph?
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The answer was yes ... I didn't see anything on StageGraphBuilder that would let you connect() to the last added node.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think StageGraphBuilder could be a little easier to use for certain common scenarios.

@@ -148,11 +148,18 @@ public void connect(@Nonnull Stage previous, @Nonnull Stage next) {
}

private String generateRefId() {
long offset = parent
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're adding stages to an already built execution, it's important to not re-use a refId.

.getExecution()
.getStages()
.stream()
.filter(i -> parent.getId().equals(i.getParentStageId()) && type == i.getSyntheticStageOwner())
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not necessary the most elegant, alternatives?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems reasonable. I can't think of a neater way off the top of my head.

onFailureStages.forEachIndexed { index, stage ->
if (index > 0) {
// all on failure stages should be run linearly
graph.connect(onFailureStages.get(index - 1), stage)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without this, we run the risk of one of the stages completing and notifying it's parent ... and completing the execution with another stage still running in parallel.

My expectation is that > 1 onFailure stages will run linearly (they did previously).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to address the root issue as well (not in this PR). The execution should not complete if anything is still in a RUNNING state and if it does that's a bug.

@ajordens ajordens force-pushed the fix-deploy-rollbacks branch 2 times, most recently from 9b86d29 to be51c8e Compare March 21, 2018 06:51
This PR handles generating a `refId` for a synthetic stage that has
pre-existing siblings.

In such cases, the `refId` should be unique and offset by the number of
siblings.

All `onFailureStages()` should be run linearly otherwise you run the
risk of a parent stage being notified prematurely.
.getExecution()
.getStages()
.stream()
.filter(i -> parent.getId().equals(i.getParentStageId()) && type == i.getSyntheticStageOwner())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems reasonable. I can't think of a neater way off the top of my head.

onFailureStages.forEachIndexed { index, stage ->
if (index > 0) {
// all on failure stages should be run linearly
graph.connect(onFailureStages.get(index - 1), stage)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to address the root issue as well (not in this PR). The execution should not complete if anything is still in a RUNNING state and if it does that's a bug.

@ajordens ajordens merged commit 18a9b26 into spinnaker:master Mar 22, 2018
@ajordens ajordens deleted the fix-deploy-rollbacks branch March 22, 2018 04:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants