Termination Protection for Kops clusters in AWS

[krisnova]

We have a use case to enable termination protection in AWS for components created with Kops.
Primarily:

• EC2 Instances
• VPC
• ASG

We would like to have an ability to deploy a termination protected cluster. All (or as many as possible) components of the cluster should be created with termination protection on in AWS.

@justinsb what are your thoughts?
@chrislovecnm what about you?

[mrichmon]

Presumably this would be useful as some separate options:

• enable termination protection on my master nodes
• enable termination protection on my worker nodes

My thinking is that cluster size will be varied frequently based on workload. And the cluster should be able to recover loss of a few worker nodes.

[chrislovecnm]

@mrichmon exactly. Also it is more that just the nodes, VPC, and the ASG.

[krisnova]

So pending @mrichmon's use case (and ours as well @chrislovecnm) - I think the big decision that needs to be made here (other than this feature in general) is :

Do we have term protection as a global flag affecting all components of the cluster?

-or-

Do we allow for component specific configuration, yes for this, no for that (IE: Master/Workers)

[chrislovecnm]

In terms of doing upgrades or other General ops. It probably would make sense to break up masters vs nodes. @justinsb any opinion on this?

[krisnova]

In the name of getting this feature through, and keeping it flexible I think we should move forward with node/master flags for term protection. @chrislovecnm can we get a priority on this - feel free to assign it to me and I can start banging it out.

[jmound]

I think EBS volume "Delete on termination" should be set to false by default. From what I see, it is currently set to true.

[krisnova]

@jmound I think there are 2 issues here... #1 Termination protection.. and #2 Delete on termination EBS volumes

I think we might need to open up a second issue for #2 (although there might be one already)

[jmound]

• I agree that they're two different issues, I was trying to add the second here. I've opened up a separate issue here Master's EBS volumes should have "delete on termination" set to False by default #1516

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle rotten
/remove-lifecycle stale

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@leoskyrocker: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[leoskyrocker]

hmm can we reopen this?

[rifelpet]

/remove-lifecycle rotten

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[SCLogo]

it would be good if we could set mfa or password or something on kops side, so we could protect cluster and configs from accidental delete