Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PodTolerationRestriction: ignore duplicates when changing pod toleration #111712

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

pacoxu
Copy link
Member

@pacoxu pacoxu commented Aug 5, 2022

What type of PR is this?

/kind bug

What this PR does / why we need it:

Reproduce steps

  1. make sure podtolerationrestriction is not enable at first.
  2. create a daemonset A with toleration {"operator":"Exists","effect":"NoSchedule"},, and pod A-1 was created
  3. enable podtolerationrestriction in all apiserver
  4. try to update the pod annotation

"reason": "FieldvalueForbidden", "message" :"Forbidden: existing toleration can not be modified except tolerationSeconds",

Workarounds

  1. just remove the pod to trigger a recreation

Which issue(s) this PR fixes:

Fixes #111745

Special notes for your reviewer:

See more in the issue reproduce steps and logics.

Does this PR introduce a user-facing change?

podtolerationrestriction: don't merge pod tolerations when existed tolerations already dups

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. kind/bug Categorizes issue or PR as related to a bug. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Aug 5, 2022
@pacoxu pacoxu closed this Aug 5, 2022
@pacoxu pacoxu reopened this Aug 8, 2022
@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Aug 8, 2022
@pacoxu pacoxu changed the title podtolerationrestriction: don't change pod toleration when update podtolerationrestriction: don't change pod toleration when there are some dups Aug 8, 2022
@pacoxu
Copy link
Member Author

pacoxu commented Aug 8, 2022

		mergedOldTolerations := tolerations.MergeTolerations(pod.Spec.Tolerations, nil)
		// if mergedOldTolerations len is different with pod.Spec.Tolerations, it means there are already some dups.
		if len(mergedOldTolerations) != len(pod.Spec.Tolerations) {
			mergedNewTolerations := tolerations.MergeTolerations(mergedOldTolerations, extraTolerations)
			// skip if no change
			if len(mergedNewTolerations) > len(mergedOldTolerations) {
				pod.Spec.Tolerations = append(pod.Spec.Tolerations, extraTolerations...)
			}
		} else {
			// if there is no dup, we can keep the old logic
			mergedNewTolerations := tolerations.MergeTolerations(pod.Spec.Tolerations, extraTolerations)
			pod.Spec.Tolerations = mergedNewTolerations
		}

@pacoxu
Copy link
Member Author

pacoxu commented Aug 8, 2022

/cc @KevinWang15 @tallclair
/assign @liggitt
/sig api-machinery

@k8s-ci-robot k8s-ci-robot added the sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. label Aug 8, 2022
@k8s-ci-robot
Copy link
Contributor

@pacoxu: GitHub didn't allow me to request PR reviews from the following users: KevinWang15.

Note that only kubernetes members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

/cc @KevinWang15 @tallclair
/assign @liggitt
/sig api-machinery

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. label Aug 8, 2022
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: pacoxu
Once this PR has been reviewed and has the lgtm label, please ask for approval from liggitt by writing /assign @liggitt in a comment. For more information see:The Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@KevinWang15
Copy link
Contributor

Given that my fix in #97850 will need to move MergeTolerations to another pkg, and that it might be considered as an api-change, I think it's okay to fix things this way, and close my PR. As long as the bug is fixed and the code owners agree on the method of the fix.

@pacoxu
Copy link
Member Author

pacoxu commented Aug 8, 2022

Given that my fix in #97850 will need to move MergeTolerations to another pkg and that it might be considered as an api-change, I think it's okay to fix things this way and close my PR as long as the bug is fixed and the code owners agree on the method of the fix.

We tried in different ways. You fix it on the validation side, and I wanted to fix it in the admission plugin.

@fedebongio
Copy link
Contributor

/triage accepted

@k8s-ci-robot k8s-ci-robot added the triage/accepted Indicates an issue or PR is ready to be actively worked on. label Aug 9, 2022
@pacoxu
Copy link
Member Author

pacoxu commented Aug 26, 2022

/priority important-longterm
This is not expected that users have to remove old pods after the apiserver configuration is changed.

@k8s-ci-robot k8s-ci-robot added priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. and removed needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Aug 26, 2022
@pacoxu
Copy link
Member Author

pacoxu commented Oct 31, 2022

Some users met this in their product cluster after enabling PodTolerationRestriction.
/cc @deads2k @derekwaynecarr
Would you take a look?

@pacoxu pacoxu changed the title podtolerationrestriction: don't change pod toleration when there are some dups PodTolerationRestriction: ignore duplicates when changing pod toleration Dec 13, 2022
@pacoxu
Copy link
Member Author

pacoxu commented Dec 13, 2022

I am not sure who is the feature owner of PodTolerationRestriction.
Can someone confirm if this works as designed or a bug?
@liggitt (sorry for disturb)

@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle stale
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Mar 13, 2023
@pacoxu
Copy link
Member Author

pacoxu commented Mar 13, 2023

/remove-lifecycle stale
ACK

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Mar 13, 2023
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle stale
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jun 11, 2023
@pacoxu
Copy link
Member Author

pacoxu commented Jul 11, 2023

/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jul 11, 2023
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle stale
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 20, 2024
@pacoxu
Copy link
Member Author

pacoxu commented Feb 18, 2024

/remove-lifecycle stale
/retest

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Feb 18, 2024
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle stale
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label May 18, 2024
@pacoxu
Copy link
Member Author

pacoxu commented May 29, 2024

/remove-lifecycle stale
/check-cla

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label May 29, 2024
@pacoxu
Copy link
Member Author

pacoxu commented May 29, 2024

/easycla

@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle stale
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Aug 27, 2024
@pacoxu
Copy link
Member Author

pacoxu commented Sep 6, 2024

/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 6, 2024
@@ -108,7 +108,16 @@ func (p *Plugin) Admit(ctx context.Context, a admission.Attributes, o admission.
}
// Final merge of tolerations irrespective of pod type.
if len(extraTolerations) > 0 {
pod.Spec.Tolerations = tolerations.MergeTolerations(pod.Spec.Tolerations, extraTolerations)
mergedOldTolerations := tolerations.MergeTolerations(pod.Spec.Tolerations, nil)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't studied the logic carefully, please correct me if I'm wrong. Intuitively, this call looks a bit strange. Can we modify the logic instead of calling the same method repeatedly with different parameters? This may make it difficult to understand the code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
7 participants