Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix kubectl drain error handling bug. #122574

Merged
merged 1 commit into from Jan 9, 2024

Conversation

brianpursley
Copy link
Member

What type of PR is this?

/kind bug

What this PR does / why we need it:

Fixed a bug where kubectl drain would consider a pod as having been deleted if an error occurs while calling the API.

This can happen if you reboot the master node(s) while draining, for example. In this case an error occurs, but the pod has not actually been deleted yet.

The problem occurs because the client-go returns an empty struct instead of nil, even when there is an error, as shown here:

func (c *pods) Get(ctx context.Context, name string, options metav1.GetOptions) (result *v1.Pod, err error) {
result = &v1.Pod{}
err = c.client.Get().
Namespace(c.ns).
Resource("pods").
Name(name).
VersionedParams(&options, scheme.ParameterCodec).
Do(ctx).
Into(result)
return
}

The kubectl drain code assumed that an error did not occur if the returned pod was not nil, which is incorrect.

This PR fixes that bug and updates the existing unit tests to use an empty struct instead of nil, matching what is actually happening. I also added a new test case to cover the case where a pod is returned, but the UID is different.

Which issue(s) this PR fixes:

Fixes kubernetes/kubectl#1532

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Fixed a bug where kubectl drain would consider a pod as having been deleted if an error occurs while calling the API.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:


@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. kind/bug Categorizes issue or PR as related to a bug. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Jan 4, 2024
@k8s-ci-robot k8s-ci-robot added area/kubectl sig/cli Categorizes an issue or PR as relevant to SIG CLI. approved Indicates a PR has been approved by an approver from all required OWNERS files. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Jan 4, 2024
@brianpursley
Copy link
Member Author

/hold for review

/cc @eddiezane
/cc @ardaguclu

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 4, 2024
@ardaguclu
Copy link
Member

/triage accepted
/priority backlog

@k8s-ci-robot k8s-ci-robot added triage/accepted Indicates an issue or PR is ready to be actively worked on. priority/backlog Higher priority than priority/awaiting-more-evidence. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Jan 4, 2024
Fixed a bug where kubectl drain would consider a pod as having been deleted if an error occurs while calling the API.
@ardaguclu
Copy link
Member

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ardaguclu, brianpursley

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@brianpursley
Copy link
Member Author

/unhold

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 4, 2024
@brianpursley
Copy link
Member Author

@ardaguclu can you lgtm if you think this is good to go?

@ardaguclu
Copy link
Member

Thanks for the fix
/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 9, 2024
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: 167b8a70e9b51f83d5cc6296edc0f413bd47398b

@k8s-ci-robot k8s-ci-robot merged commit 56ecdad into kubernetes:master Jan 9, 2024
14 checks passed
@k8s-ci-robot k8s-ci-robot added this to the v1.30 milestone Jan 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/kubectl cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/backlog Higher priority than priority/awaiting-more-evidence. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/cli Categorizes an issue or PR as relevant to SIG CLI. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

kubectl drain ignores terminating pods when reboot master node
3 participants