-
Notifications
You must be signed in to change notification settings - Fork 38.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
kubelet: retry pod sandbox creation when containers were never created #79451
Conversation
If kubelet never gets past sandbox creation (i.e., never attempted to create containers for a pod), it should retry the sandbox creation on failure, regardless of the restart policy of the pod.
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: yujuhong The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
The regression was introduced in 1.13. We need to patch 1.15, 1.14, and 1.13. |
The original change was introduced in #68980 /cc @derekwaynecarr |
@@ -472,11 +472,15 @@ func (m *kubeGenericRuntimeManager) computePodActions(pod *v1.Pod, podStatus *ku | |||
// If we need to (re-)create the pod sandbox, everything will need to be | |||
// killed and recreated, and init containers should be purged. | |||
if createPodSandbox { | |||
if !shouldRestartOnFailure(pod) && attempt != 0 { | |||
if !shouldRestartOnFailure(pod) && attempt != 0 && len(podStatus.ContainerStatuses) != 0 { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we also add check len(podStatus.InitContainerStatuses) != 0
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a kubelet's internal type; there's no initcontianerstatuses.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The unit tests added show that this would work for init containers too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/lgtm
modulo the open question about init containers this looks good to me! Thanks for the quick fix and for adding a test case.
/retest |
/cc @dashpole |
/lgtm |
/retest Don't think this change would affect pull-kubernetes-kubemark-e2e-gce-big |
…51-upstream-release-1.14 Automated cherry pick of #79451: kubelet: retry pod sandbox creation when containers were
…51-upstream-release-1.15 Automated cherry pick of #79451: kubelet: retry pod sandbox creation when containers were
…51-upstream-release-1.13 Automated cherry pick of #79451: kubelet: retry pod sandbox creation when containers were
from what version of kubernetes is this fix part of? |
1.16+ |
If kubelet never gets past sandbox creation (i.e., never attempted to
create containers for a pod), it should retry the sandbox creation on
failure, regardless of the restart policy of the pod.
What type of PR is this?
/kind bug
What this PR does / why we need it:
Which issue(s) this PR fixes:
Fixes #79398
Special notes for your reviewer:
Does this PR introduce a user-facing change?: