Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add wildcard tolerations to kube-proxy #56589

Merged
merged 2 commits into from Nov 30, 2017
Merged

Add wildcard tolerations to kube-proxy #56589

merged 2 commits into from Nov 30, 2017

Conversation

rohitagarwal003
Copy link
Member

@rohitagarwal003 rohitagarwal003 commented Nov 29, 2017 •
edited

  • Add wildcard tolerations to kube-proxy.
  • Add nvidia.com/gpu toleration to nvidia-gpu-device-plugin.

Related to #55080 and #44445.

/kind bug
/priority critical-urgent
/sig scheduling

Release note:

kube-proxy addon tolerates all NoExecute and NoSchedule taints by default.

/assign @davidopp @bsalamat @vishh @jiayingz

@rohitagarwal003
nvidia-gpu-device-plugin daemonset should tolerate nvidia.com/gpu taint.
d734174 
It is expected that nodes with extended resources attached will be
tainted with the resouce name, so that we can create dedicated nodes.
If ExtendedResourceToleration admission controller is enabled, pods
requesting such resources will automatically tolerate such taints.
nvidia-gpu-device-plugin daemonset doesn't request such resources but
still needs to run on such nodes, so it needs this toleration.
@rohitagarwal003
Add wildcard tolerations to kube-proxy.
ad05928 
fluend-gcp already has these tolerations. kube-proxy when it runs as a
static pod gets wildcard `NoExecute` toleration (all static pods get
that). So, added the same toleration to kube-proxy when it runs as a
daemonset. Also added wildcard `NoSchedule` toleration to kube-proxy.
@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Nov 29, 2017
@rohitagarwal003
Copy link
Member Author

/cc @MrHohn @bowei @gmarek

@vishh vishh added the kind/bug Categorizes issue or PR as related to a bug. label Nov 29, 2017
@vishh
Copy link
Contributor

vishh commented Nov 29, 2017

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 29, 2017
@vishh
Copy link
Contributor

vishh commented Nov 29, 2017

cc @mikedanese for approval

MrHohn
MrHohn approved these changes Nov 29, 2017
View reviewed changes
Copy link
Member

@MrHohn MrHohn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kube-proxy part LGTM.
/lgtm

@rohitagarwal003
Copy link
Member Author

/retest

@jiayingz
Copy link
Contributor

/lgtm

@mikedanese
Copy link
Member

squash?

mikedanese
mikedanese reviewed Nov 29, 2017
View reviewed changes
cluster/addons/fluentd-gcp/fluentd-gcp-ds.yaml
@@ -107,7 +107,6 @@ spec:
effect: "NoSchedule"
- operator: "Exists"
effect: "NoExecute"
#TODO: remove this toleration once #44445 is properly fixed.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why remove this comment? The issue hasn't closed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't want this toleration to be removed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't is different than we never will. TODO indicates we don't but we might want to eventually. Is this issue obsolete? Should we close it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#44445 contains multiple issues.

  • The title of the issue: Improvement: fluentd-gcp to get same toleration as kube-proxy will be fixed by this PR.
  • There was a regression in the daemonset controller mentioned in the bug which was fixed a while back.
  • I guess the only thing that is not fixed are the three comments starting from Improvement: fluentd-gcp to get same toleration as kube-proxy #44445 (comment) (users not being able to modify system addons on managed services like GKE and so can't use NoSchedule or NoExecute taints if these addons don't tolerate them because adding such taints make the nodes not have these "required" addons)

But even if we fix that (allow users to modify the toleration of system addons). I think the default should still be that these addons tolerate all taints. If users really want they can use the ability to modify the toleration of system addons to remove these wildcard tolerations.

Also, we need system addons that run on every node to have wildcard NoSchedule toleration for issue #55080, PR #55839.

cc @vishh

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. The expected behavior is that all system addons that are expected to run on all GKE nodes should tolerate all taints and effects. Certain addons like GPU plugins need to tolerate GPU specific taints only.
This I feel is probably slightly different from #44445 where if a user taints all GKE nodes then cluster level system addons will not run at all. This is a separate feature and is not tied to the comment at all.
@mikedanese thoughts?

@rohitagarwal003
Copy link
Member Author

rohitagarwal003 commented Nov 29, 2017 •
edited

squash?

Those commits are doing two different things. The commit messages have more detail.

@davidopp davidopp added this to the v1.9 milestone Nov 30, 2017
@k8s-github-robot
Copy link

[MILESTONENOTIFIER] Milestone Pull Request Needs Attention

@MrHohn @bsalamat @davidopp @jiayingz @mikedanese @mindprince @vishh @kubernetes/sig-scheduling-misc

Action required: During code freeze, pull requests in the milestone should be in progress.
If this pull request is not being actively worked on, please remove it from the milestone.
If it is being worked on, please add the status/in-progress label so it can be tracked with other in-flight pull requests.

Note: This pull request is marked as priority/critical-urgent, and must be updated every 1 day during code freeze.

Example update:

ACK.  In progress
ETA: DD/MM/YYYY
Risks: Complicated fix required
Pull Request Labels
  • sig/scheduling: Pull Request will be escalated to these SIGs if needed.
  • priority/critical-urgent: Never automatically move pull request out of a release milestone; continually escalate to contributor and SIG through all available channels.
  • kind/bug: Fixes a bug discovered during the current release.
Help

@mikedanese
Copy link
Member

/approve

@k8s-github-robot k8s-github-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 30, 2017
@k8s-github-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jiayingz, mikedanese, mindprince, MrHohn, vishh

Associated issue: 55080

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these OWNERS Files:

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

@k8s-github-robot
Copy link

/test all [submit-queue is verifying that this PR is safe to merge]

@k8s-github-robot
Copy link

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here.

@k8s-github-robot k8s-github-robot merged commit d88ce26 into kubernetes:master Nov 30, 2017
@derekperkins derekperkins mentioned this pull request Jul 12, 2018
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mikedanese mikedanese mikedanese left review comments

@vishh vishh vishh left review comments

@MrHohn MrHohn MrHohn approved these changes

@bowei bowei Awaiting requested review from bowei

@gmarek gmarek Awaiting requested review from gmarek

Assignees

@mikedanese mikedanese

@vishh vishh

@davidopp davidopp

@MrHohn MrHohn

@bsalamat bsalamat

@jiayingz jiayingz

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. milestone/needs-attention priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Milestone
v1.9
Development

Successfully merging this pull request may close these issues.

None yet
9 participants
@rohitagarwal003 @vishh @jiayingz @mikedanese @k8s-github-robot @MrHohn @davidopp @bsalamat @k8s-ci-robot