New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EKS cluster creation should deal with rolled back stacks #2511

Merged
merged 4 commits into from Dec 19, 2018

Conversation

Projects
None yet
5 participants
@hekonsek
Copy link
Contributor

hekonsek commented Dec 14, 2018

If EKS cluster creation process is interrupted or if cluster with given name already exists, there will be CloudFormation stack in ROLLBACK_COMPLETE state left. Such dead stack prevents eksctl from creating cluster with the same name. This is common activity then to remove stacks like this and this function performs this action.

This PR detects such dead stacks and deletes them before creating the cluster with eksctl.

Also we should try to detect that cluster with given name exists before running eksctl, otherwise we end up with CloudFormation stack being initialized and failing after some time. If we know that cluster with given name already exists, we should just detect this fact and inform user about it as soon as possible. This PR implements this as well.

@jenkins-x-bot jenkins-x-bot requested review from jstrachan and markawm Dec 14, 2018

@jenkins-x-bot

This comment has been minimized.

Copy link
Contributor

jenkins-x-bot commented Dec 14, 2018

Hi @hekonsek. Thanks for your PR.

I'm waiting for a jenkins-x member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@hekonsek

This comment has been minimized.

Copy link
Contributor

hekonsek commented Dec 17, 2018

False negative? :) Maybe we should re-run tests.

@ccojocar

This comment has been minimized.

Copy link
Member

ccojocar commented Dec 17, 2018

/ok-to-test

@ccojocar

This comment has been minimized.

Copy link
Member

ccojocar commented Dec 17, 2018

/retest

@ccojocar

This comment has been minimized.

Copy link
Member

ccojocar commented Dec 17, 2018

/test bdd

@jenkins-x-bot jenkins-x-bot removed the lgtm label Dec 17, 2018

@ccojocar

This comment has been minimized.

Copy link
Member

ccojocar commented Dec 17, 2018

/test bdd

if clusterExists {
logger.Infof("EKS cluster %s already exists.", util.ColorInfo(flags.ClusterName))
return nil
} else {

This comment has been minimized.

@houndci-bot

houndci-bot Dec 17, 2018

Collaborator

if block ends with a return statement, so drop this else and outdent its block

return err
}

func EksctlStackName(clusterName string) string {

This comment has been minimized.

@houndci-bot

houndci-bot Dec 17, 2018

Collaborator

exported function EksctlStackName should have comment or be unexported

return false, nil
}

func CleanUpObsoleteEksClusterStack(clusterName string, profile string, region string) error {

This comment has been minimized.

@houndci-bot

houndci-bot Dec 17, 2018

Collaborator

exported function CleanUpObsoleteEksClusterStack should have comment or be unexported

return false, nil
}

/**

This comment has been minimized.

@houndci-bot

houndci-bot Dec 17, 2018

Collaborator

comment on exported function EksClusterObsoleteStackExists should be of the form "EksClusterObsoleteStackExists ..."

"strings"
)

func EksClusterExists(clusterName string, profile string, region string) (bool, error) {

This comment has been minimized.

@houndci-bot

houndci-bot Dec 17, 2018

Collaborator

exported function EksClusterExists should have comment or be unexported

@pmuir

This comment has been minimized.

Copy link
Member

pmuir commented Dec 18, 2018

/test bdd

@pmuir

This comment has been minimized.

Copy link
Member

pmuir commented Dec 18, 2018

/test bdd

@ccojocar

This comment has been minimized.

Copy link
Member

ccojocar commented Dec 18, 2018

/lgtm
/test bdd

@jenkins-x-bot jenkins-x-bot added the lgtm label Dec 18, 2018

@ccojocar

This comment has been minimized.

Copy link
Member

ccojocar commented Dec 18, 2018

/test this

@jenkins-x-bot jenkins-x-bot removed the lgtm label Dec 19, 2018

@ccojocar

This comment has been minimized.

Copy link
Member

ccojocar commented Dec 19, 2018

/test this

@ccojocar

This comment has been minimized.

Copy link
Member

ccojocar commented Dec 19, 2018

/lgtm

@jenkins-x-bot jenkins-x-bot added the lgtm label Dec 19, 2018

@jenkins-x-bot

This comment has been minimized.

Copy link
Contributor

jenkins-x-bot commented Dec 19, 2018

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ccojocar

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@jenkins-x-bot jenkins-x-bot merged commit 47eccbd into jenkins-x:master Dec 19, 2018

2 checks passed

serverless-jenkins succeeded
tide In merge pool.
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment