OCPBUGS-9274: canary: Tolerate infra node NoExecute taint #932

Miciah · 2023-05-18T22:02:05Z

Configure the canary daemonset to tolerate a "node-role.kubernetes.io/infra" node taint that specifies the NoExecute effect (or any other effect).

Before this change, the canary daemonset tolerated the "node-role.kubernetes.io/infra" node taint only if it specified the NoSchedule effect. However, some cluster admins taint infra nodes using both NoSchedule as well as NoExecute, in which case canary pods would not be scheduled on these nodes. This inconsistency caused some confusion. With this change, canary pods should be scheduled on infra nodes whether or not they are tainted using the NoExecute effect.

assets/canary/daemonset.yaml: Tolerate the "node-role.kubernetes.io/infra" taint irrespective of the specified effect.
pkg/manifests/bindata.go: Regenerate.
pkg/operator/controller/canary/daemonset_test.go (TestDesiredCanaryDaemonSet): Expect Effect to be unspecified on the toleration.
(TestCanaryDaemonsetChanged): Add a test case to verify that canaryDaemonSetChanged correctly detects the change and updates the daemonset if the specified effect changes on the toleration.

Configure the canary daemonset to tolerate a "node-role.kubernetes.io/infra" node taint that specifies the "NoExecute" effect (or any other effect). Before this commit, the canary daemonset tolerated the "node-role.kubernetes.io/infra" node taint only if it specified the "NoSchedule" effect. However, some cluster admins taint infra nodes using both "NoSchedule" as well as "NoExecute", in which case canary pods would not be scheduled on these nodes. This inconsistency caused some confusion. With this commit, canary pods should be scheduled on infra nodes whether or not they are tainted using the "NoExecute" effect. This commit fixes OCPBUGS-9274. https://issues.redhat.com/browse/OCPBUGS-9274 * assets/canary/daemonset.yaml: Tolerate the "node-role.kubernetes.io/infra" taint irrespective of the specified effect. * pkg/manifests/bindata.go: Regenerate. * pkg/operator/controller/canary/daemonset_test.go (TestDesiredCanaryDaemonSet): Expect "Effect" to be unspecified on the toleration. (TestCanaryDaemonsetChanged): Add a test case to verify that canaryDaemonSetChanged correctly detects the change and updates the daemonset if the specified effect changes on the toleration.

openshift-ci-robot · 2023-05-18T22:02:12Z

@Miciah: This pull request references Jira Issue OCPBUGS-9274, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.14.0) matches configured target version for branch (4.14.0)
bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @ShudiLi

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

Configure the canary daemonset to tolerate a "node-role.kubernetes.io/infra" node taint that specifies the NoExecute effect (or any other effect).

Before this change, the canary daemonset tolerated the "node-role.kubernetes.io/infra" node taint only if it specified the NoSchedule effect. However, some cluster admins taint infra nodes using both NoSchedule as well as NoExecute, in which case canary pods would not be scheduled on these nodes. This inconsistency caused some confusion. With this change, canary pods should be scheduled on infra nodes whether or not they are tainted using the NoExecute effect.

assets/canary/daemonset.yaml: Tolerate the "node-role.kubernetes.io/infra" taint irrespective of the specified effect.

pkg/manifests/bindata.go: Regenerate.

pkg/operator/controller/canary/daemonset_test.go (TestDesiredCanaryDaemonSet): Expect Effect to be unspecified on the toleration.
(TestCanaryDaemonsetChanged): Add a test case to verify that canaryDaemonSetChanged correctly detects the change and updates the daemonset if the specified effect changes on the toleration.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

candita · 2023-05-24T15:07:11Z

/assign

candita · 2023-05-30T15:14:32Z

/retest-required

candita · 2023-05-31T18:12:21Z

Hypershift error:
2023-05-30T15:22:47Z INFO Detected Issuer URL {"issuer": "https://hypershift-ci-1-oidc.s3.us-east-1.amazonaws.com/1b0ddf9389406da45310-mgmt"}
2023-05-30T15:22:47Z ERROR Failed to create cluster {"error": "failed to create iam: LimitExceeded: Cannot exceed quota for OpenIdConnectProvidersPerAccount: 100\n\tstatus

/retest-required

candita · 2023-05-31T20:44:44Z

Seeing this repeatedly, wonder if it is transient:

[controller-runtime] log.SetLogger(...) was never called, logs will not be displayed:

/retest-required

Miciah · 2023-06-01T02:11:10Z

e2e-aws-operator failed because kube-apiserver and ovnkube-master failed to roll out some pods.
/test e2e-aws-operator

candita · 2023-06-01T18:46:26Z

/lgtm
/approve

openshift-ci · 2023-06-01T18:46:56Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: candita

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [candita]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Miciah · 2023-06-01T19:48:01Z

/hold
#939 is top priority; make sure it merges first.

Miciah · 2023-07-06T16:23:47Z

/hold cancel
/test all

Miciah · 2023-07-06T17:04:36Z

e2e-aws-operator, e2e-aws-ovn, e2e-aws-ovn-serial, e2e-aws-ovn-single-node, and e2e-aws-ovn-upgrade failed because teh build cluster had no available nodes.
/test e2e-aws-operator

openshift-ci-robot · 2023-07-06T17:31:35Z

/retest-required

Remaining retests: 0 against base HEAD 0cdc132 and 2 for PR HEAD aa400d8 in total

openshift-ci · 2023-07-06T20:02:12Z

@Miciah: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-gcp-ovn	`aa400d8`	link	false	`/test e2e-gcp-ovn`
ci/prow/e2e-azure-ovn	`aa400d8`	link	false	`/test e2e-azure-ovn`
ci/prow/e2e-aws-ovn-single-node	`aa400d8`	link	false	`/test e2e-aws-ovn-single-node`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

candita · 2023-07-19T17:22:30Z

/retest-required

openshift-ci-robot · 2023-07-19T19:36:53Z

@Miciah: Jira Issue OCPBUGS-9274: All pull requests linked via external trackers have merged:

openshift/cluster-ingress-operator#932

Jira Issue OCPBUGS-9274 has been moved to the MODIFIED state.

In response to this:

Configure the canary daemonset to tolerate a "node-role.kubernetes.io/infra" node taint that specifies the NoExecute effect (or any other effect).

Before this change, the canary daemonset tolerated the "node-role.kubernetes.io/infra" node taint only if it specified the NoSchedule effect. However, some cluster admins taint infra nodes using both NoSchedule as well as NoExecute, in which case canary pods would not be scheduled on these nodes. This inconsistency caused some confusion. With this change, canary pods should be scheduled on infra nodes whether or not they are tainted using the NoExecute effect.

assets/canary/daemonset.yaml: Tolerate the "node-role.kubernetes.io/infra" taint irrespective of the specified effect.

pkg/manifests/bindata.go: Regenerate.

pkg/operator/controller/canary/daemonset_test.go (TestDesiredCanaryDaemonSet): Expect Effect to be unspecified on the toleration.
(TestCanaryDaemonsetChanged): Add a test case to verify that canaryDaemonSetChanged correctly detects the change and updates the daemonset if the specified effect changes on the toleration.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot added the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label May 18, 2023

openshift-ci bot requested review from ShudiLi, alebedev87 and candita May 18, 2023 22:02

openshift-ci bot assigned candita May 24, 2023

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jun 1, 2023

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 1, 2023

openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 1, 2023

openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 6, 2023

openshift-merge-robot merged commit 80cfabf into openshift:master Jul 19, 2023
11 of 14 checks passed

Miciah mentioned this pull request Jul 19, 2023

NE-1269: Replace bindata using embed #905

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OCPBUGS-9274: canary: Tolerate infra node NoExecute taint #932

OCPBUGS-9274: canary: Tolerate infra node NoExecute taint #932

Miciah commented May 18, 2023

openshift-ci-robot commented May 18, 2023

candita commented May 24, 2023

candita commented May 30, 2023

candita commented May 31, 2023

candita commented May 31, 2023

Miciah commented Jun 1, 2023

candita commented Jun 1, 2023

openshift-ci bot commented Jun 1, 2023

Miciah commented Jun 1, 2023

Miciah commented Jul 6, 2023

Miciah commented Jul 6, 2023

openshift-ci-robot commented Jul 6, 2023

openshift-ci bot commented Jul 6, 2023 •

edited

candita commented Jul 19, 2023

openshift-ci-robot commented Jul 19, 2023

OCPBUGS-9274: canary: Tolerate infra node NoExecute taint #932

OCPBUGS-9274: canary: Tolerate infra node NoExecute taint #932

Conversation

Miciah commented May 18, 2023

openshift-ci-robot commented May 18, 2023

candita commented May 24, 2023

candita commented May 30, 2023

candita commented May 31, 2023

candita commented May 31, 2023

Miciah commented Jun 1, 2023

candita commented Jun 1, 2023

openshift-ci bot commented Jun 1, 2023

Miciah commented Jun 1, 2023

Miciah commented Jul 6, 2023

Miciah commented Jul 6, 2023

openshift-ci-robot commented Jul 6, 2023

openshift-ci bot commented Jul 6, 2023 • edited

candita commented Jul 19, 2023

openshift-ci-robot commented Jul 19, 2023

openshift-ci bot commented Jul 6, 2023 •

edited