KEP for critical container feature

[CsatariGergely]

Signed-off-by: Gergely Csatari gergely.csatari@nokia.com

[k8s-ci-robot]

Hi @CsatariGergely. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: CsatariGergely
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: derekwaynecarr

If they are not already assigned, you can assign the PR to them by writing /assign @derekwaynecarr in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• keps/sig-node/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[CsatariGergely]

/cc: @libesz @janosi

[derekwaynecarr]

Thank you for proposing this idea.

Can we discuss this in more detail at a future SIG node meeting so we understand the problems and goals?

[libesz]

@derekwaynecarr You can find some use-cases here: kubernetes/kubernetes#40908
Pod replacement in my mind is the process what is done when all the container including the infra (pause) is terminated and re-created from the PodSpecs.
The idea was that we don't want to introduce a new procedure to "restart" a pod in-place (which i.e. preserves the emptyDir volumes, etc.), instead use the existing terminate and create procedures of the Pod, done by kubelet. Hope our assumptions are correct 🤞 .

[CsatariGergely]

@derekwaynecarr Thanks for the comments. To join to the sig-node meeting this week is a bit difficult from CET timezone. Next week I will be closer timezone wise and I will try to join.

[CsatariGergely]

Notes from sig-node meeting at 02.04.2019:

• Sidecar containers KEP1, KEP2 is doing similar in a sense that some containers are differently handled from the pods lifececyle point of view
• We should check if this provides a solution to our problem

[libesz]

Notes from sig-node meeting at 02.04.2019:

* Sidecar containers [KEP1](https://github.com/kubernetes/enhancements/pull/919), [KEP2](https://github.com/kubernetes/enhancements/blob/master/keps/sig-apps/sidecarcontainers.md) is doing similar in a sense that some containers are differently handled from the pods lifececyle point of view

* We should check if this provides a solution to our problem

Just checked the other KEPs. I believe those are covering different use-cases around lifecycle management, while this one wants to cover failure/recovery scenarios. I would consider this one as independent from specification/implementation PoV.

[justaugustus]

/ok-to-test

[InAnimaTe]

This is a great KEP and I'd love to see it get approved. @CsatariGergely @derekwaynecarr what specifically is being waited for prior to approval?

[CsatariGergely]

Thanks @InAnimaTe :) I would be also interested to figure out what is needed to approve this. I'm available in Barcelona to discuss if that helps.

[thockin]

I just want to say that I have heard this request many times, and while I continue to assert that "you're doing it wrong" it seems common enough to consider. I urge you to look for the simplest design that satisfies the preponderance of users.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[bitva77]

/remove-lifecycle stale

We've recently run across a potential bug in dotnet core containers that an enhancement like this would help us workaround downtime in our service (restarting containers is not enough - the pod needs to be restarted. I suspect an issue more with Docker and dotnet).

[byxorna]

This would be an amazingly useful feature for usecases where the pod health is dependent on the health of all component containers - a failure in one container invalidates the health of all others in the pod (yes, i know that isnt ideal SOA design, but legacy is going to legacy). I have tooling currently emulates this behavior external to k8s (via a k8s aware init replacement), but it is kludgy and would be much nicer if this support was mainlined :)

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[bitva77]

/remove-lifecycle

still love to see this :)

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[bitva77]

/remove-lifecycle rotten

[InAnimaTe]

I think myself and others would still really like to see this get Merged in. Is there really anything else needed here to get the ball rolling? Can I help in any way like with documentation or design or something? I'd love to chat more about what this looks like and why it should exist.

[thockin]

This overlaps a lot with other pod-lifecycle issues. I think we need holistic review of the topic.

kubernetes/kubernetes#88886
kubernetes/kubernetes#87801
kubernetes/kubernetes#87016
kubernetes/kubernetes#88257
kubernetes/kubernetes#65502
kubernetes/kubernetes#80744
kubernetes/kubernetes#40908
kubernetes/kubernetes#25908
The pod-restart one which I can't find right now
and so forth

@derekwaynecarr has expressed concerns here and the more I look, the more I agree.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[bitva77]

/remove-lifecycle stale

By default, just restart the container as it does today. But an option to replace the pod would be super.

[thockin]

Hi,

I know this KEP has languished. IMO this needs to be part of a larger pod-lifecycle effort. I think it is likely to be highly entangled with a bunch of other proposals and ideas, and we should probably not be patching each one individually.

e.g. kubernetes/community#2342 kubernetes/kubernetes#25908 and others that I can't find links for right now.

I do support the idea expressed here, but I don't think we should do it piecemeal.

[cnmcavoy]

@thockin I saw your comment about this needing a larger review of the pod-lifecycle. Do you have any suggestions for what the next steps for such an effort would be?

My use-case for this is that we have a service which measures the time it takes an ingress to discover the pod the service is running in and reconfigure itself to serve traffic (e.g nginx reload). After the pod detects the ingress is up to date, it reports the metric and kills itself, so it can be rescheduled with a new pod ip. Unfortunately there is not a clean way to do this behavior currently: if a container exits abnormally, kubelet restarts the container in place and the pod retains its ip/network state. As a terrible workaround, we have the service exit by exceeding it's ephemeral disk quota, which causes kubelet to evict, delete the pod, then reschedule it. This KEP would allow us to exit cleanly and let kubelet restart the entire pod.

I'm interested in this enhancement and want to contribute, but It is unclear to me what the next steps are.

[dbsanfte]

It feels like being able to set 'restartPolicy: Never' on a ReplicaSet container (and have it be honoured) is a cleaner solution to the use case presented in this issue, and as a bonus it requires less code change, and requires no spec change.

This is already logged here if there's any interest (the issue badly needs reopening, imo):

kubernetes/kubernetes#24725

[ehashman]

Hi all,

This PR has not seen any updates in nearly two years, and I see there are some concerns about the approach. The KEP templates have changed pretty significantly since this PR was opened, so it is not in a state where it is mergeable. Hence, I am going to close this, and if someone has time to pick up this idea and ferry it through the KEP process and propose a new PR, that would be much appreciated.