Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changed IsCriticalPod to return true in case of static pods #80491

Merged
merged 1 commit into from Aug 8, 2019

Conversation

@hpandeycodeit
Copy link
Member

commented Jul 23, 2019

What type of PR is this?
/kind bug

What this PR does / why we need it:

Currently, If the static pods is critical, the kubelet will not realize it, since it's priority value determines whether the pod is critical or not. So SIG Node decided to consider all static pods as critical regardless of their PodSpec.PriorityClassName.

Which issue(s) this PR fixes:

Fixes #80489

Does this PR introduce a user-facing change?:

Kubelet considers all static pods as critical. Static pods pass kubelet admission even if a node does not have enough resources. Users must ensure that they account for resources when creating static pods.
@hpandeycodeit

This comment has been minimized.

Copy link
Member Author

commented Jul 23, 2019

/sig scheduling
/sig node

@hpandeycodeit

This comment has been minimized.

Copy link
Member Author

commented Jul 23, 2019

/retest

@draveness
Copy link
Member

left a comment

/priority important-soon

@mattjmcnaughton
Copy link
Contributor

left a comment

Thanks for this pr :)

I'm curious to see the results of the test suite after running the two hack scripts below. You may have an import cycle, in which case we may need to rethink the approach slightly. We'll know for sure after re-running the test suite after running the hack scripts.

@@ -23,6 +23,7 @@ import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
utilfeature "k8s.io/apiserver/pkg/util/feature"
kubeapi "k8s.io/kubernetes/pkg/apis/core"
kubepod "k8s.io/kubernetes/pkg/kubelet/pod"

This comment has been minimized.

Copy link
@mattjmcnaughton

mattjmcnaughton Jul 24, 2019

Contributor

Please run ./hack/update-bazel.sh to add this new dependency to the proper BUILD.bazel file.

This comment has been minimized.

Copy link
@hpandeycodeit

hpandeycodeit Jul 24, 2019

Author Member

I ran the above hack scripts. Doesn't look like it's helping with the import cycle though :(

@@ -146,6 +147,9 @@ func (sp SyncPodType) String() string {
// or equal to SystemCriticalPriority. Both the default scheduler and the kubelet use this function
// to make admission and scheduling decisions.
func IsCriticalPod(pod *v1.Pod) bool {
if kubepod.IsStaticPod(pod){

This comment has been minimized.

Copy link
@mattjmcnaughton

mattjmcnaughton Jul 24, 2019

Contributor

Please run ./hack/update-gofmt.sh :)

@bsalamat
Copy link
Member

left a comment

Looks good. You may have to move IsCriticalPod to pkg/kubelet/pod, or another similar move to remove the circular dependency.
Also, please make sure to add adequate tests to cover the new logic.

@k8s-ci-robot k8s-ci-robot added size/M and removed size/XS labels Jul 24, 2019

@hpandeycodeit

This comment has been minimized.

Copy link
Member Author

commented Jul 24, 2019

circular dependency

Thanks @bsalamat! I made the needed change to remove the import cycle.

Name: "pod5",
Namespace: "kube-system",
Annotations: map[string]string{
"scheduler.alpha.kubernetes.io/critical-pod": "file",

This comment has been minimized.

Copy link
@bsalamat

bsalamat Jul 25, 2019

Member

If you meant to add a static pod, the key should have been ConfigSourceAnnotationKey. What you have used is the deprecated critical annotation key.

@bsalamat bsalamat added this to the v1.16 milestone Jul 25, 2019

@hpandeycodeit

This comment has been minimized.

Copy link
Member Author

commented Jul 25, 2019

/test pull-kubernetes-e2e-gce-100-performance

@bsalamat
Copy link
Member

left a comment

/lgtm

Please go ahead and squash commits.
Thanks, @hpandeycodeit!

@hpandeycodeit

This comment has been minimized.

Copy link
Member Author

commented Jul 29, 2019

/test pull-kubernetes-e2e-gce-device-plugin-gpu

@bsalamat
Copy link
Member

left a comment

/lgtm

/assign @dchen1107
for reviewing and approval

@derekwaynecarr

This comment has been minimized.

Copy link
Member

commented Aug 6, 2019

I am not necessarily adverse to this. I know I was out of the SIG node meeting where this was discussed as a result of PTO. I am wondering why the static pod author is not able to set priorityClassName on their manifest to system-cluster-critical ? we have been doing this, and i had not seen an issue with that approach.

@bsalamat

This comment has been minimized.

Copy link
Member

commented Aug 6, 2019

I am not necessarily adverse to this. I know I was out of the SIG node meeting where this was discussed as a result of PTO. I am wondering why the static pod author is not able to set priorityClassName on their manifest to system-cluster-critical ? we have been doing this, and i had not seen an issue with that approach.

@derekwaynecarr They can set priorityClassName in their podspec, but the integer value of the priority will not be resolved by time a static pod goes through admission logic of Kubelet. Please see #80489 for more info.

@hpandeycodeit hpandeycodeit force-pushed the hpandeycodeit:static_pod_priority branch from 9bea463 to c05d506 Aug 7, 2019

@dchen1107

This comment has been minimized.

Copy link
Member

commented Aug 8, 2019

@derekwaynecarr We discussed this at SIG Node, and I documented what we discussed at the original issue: #80203 (comment) I believe I also recorded the meeting. Please let me know if you had a second thought on it. Otherwise, I plan to approve the pr. Thanks!

@dchen1107

This comment has been minimized.

Copy link
Member

commented Aug 8, 2019

/lgtm

Will wait for @derekwaynecarr before giving the final approval.

@k8s-ci-robot k8s-ci-robot added the lgtm label Aug 8, 2019

@derekwaynecarr

This comment has been minimized.

Copy link
Member

commented Aug 8, 2019

@bsalamat thanks for the missing detail.

/approve

@draveness

This comment has been minimized.

Copy link
Member

commented Aug 8, 2019

/retest

@dchen1107

This comment has been minimized.

Copy link
Member

commented Aug 8, 2019

/approve

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

commented Aug 8, 2019

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dchen1107, derekwaynecarr, hpandeycodeit

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit 806ced3 into kubernetes:master Aug 8, 2019

23 checks passed

cla/linuxfoundation hpandeycodeit authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-conformance-image-test Skipped.
pull-kubernetes-cross Skipped.
pull-kubernetes-dependencies Job succeeded.
Details
pull-kubernetes-e2e-gce Job succeeded.
Details
pull-kubernetes-e2e-gce-100-performance Job succeeded.
Details
pull-kubernetes-e2e-gce-csi-serial Skipped.
pull-kubernetes-e2e-gce-device-plugin-gpu Job succeeded.
Details
pull-kubernetes-e2e-gce-iscsi Skipped.
pull-kubernetes-e2e-gce-iscsi-serial Skipped.
pull-kubernetes-e2e-gce-storage-slow Skipped.
pull-kubernetes-godeps Skipped.
pull-kubernetes-integration Job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce-big Job succeeded.
Details
pull-kubernetes-local-e2e Skipped.
pull-kubernetes-node-e2e Job succeeded.
Details
pull-kubernetes-node-e2e-containerd Job succeeded.
Details
pull-kubernetes-typecheck Job succeeded.
Details
pull-kubernetes-verify Job succeeded.
Details
pull-publishing-bot-validate Skipped.
tide In merge pool.
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.