Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[release-4.12] OCPBUGS-10933: Fix cleanup failure on WMCO upgrade #1527

Conversation

openshift-cherrypick-robot

This is an automated cherry-pick of #1511

/assign mansikulkarni96

This commit rearranges the order of WMCO instance
deconfiguration operations.
It also updates the directory removal operation to
delete the ksDir files excluding the WICD service
related files. This allows us to retry
reconciliation in case of a deletion failure,
resulting in WICD cleanup cmd being called.
k8sDir which only contains the WICD files is deleted
as the last step in the directory cleanup process with
best effort, logging any errors. At this point we don't
want to retry reconciliation as the WICD files would
have been deleted.
@openshift-ci-robot
Copy link

@openshift-cherrypick-robot: Jira Issue OCPBUGS-10709 has been cloned as Jira Issue OCPBUGS-10933. Will retitle bug to link to clone.
/retitle [release-4.12] OCPBUGS-10933: Fix cleanup failure on WMCO upgrade

In response to this:

This is an automated cherry-pick of #1511

/assign mansikulkarni96

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot changed the title [release-4.12] OCPBUGS-10709: Fix cleanup failure on WMCO upgrade [release-4.12] OCPBUGS-10933: Fix cleanup failure on WMCO upgrade Mar 27, 2023
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Mar 27, 2023
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Mar 27, 2023

@openshift-cherrypick-robot: No Bugzilla bug is referenced in the title of this pull request.
To reference a bug, add 'Bug XXX:' to the title of this pull request and request another bug refresh with /bugzilla refresh.

In response to this:

[release-4.12] OCPBUGS-10933: Fix cleanup failure on WMCO upgrade

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci-robot openshift-ci-robot added the jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. label Mar 27, 2023
@openshift-ci-robot
Copy link

@openshift-cherrypick-robot: This pull request references Jira Issue OCPBUGS-10933, which is invalid:

  • expected dependent Jira Issue OCPBUGS-10709 to be in one of the following states: VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), but it is ON_QA instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

This is an automated cherry-pick of #1511

/assign mansikulkarni96

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@mansikulkarni96
Copy link
Member

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Mar 27, 2023
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Mar 27, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aravindhp, openshift-cherrypick-robot

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 27, 2023
@mtnbikenc
Copy link
Member

/jira refresh

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Mar 29, 2023
@openshift-ci-robot
Copy link

@mtnbikenc: This pull request references Jira Issue OCPBUGS-10933, which is valid.

6 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.12.z) matches configured target version for branch (4.12.z)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)
  • dependent bug Jira Issue OCPBUGS-10709 is in the state Verified, which is one of the valid states (VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE))
  • dependent Jira Issue OCPBUGS-10709 targets the "4.13.0" version, which is one of the valid target versions: 4.13.0
  • bug has dependents

Requesting review from QA contact:
/cc @rrasouli

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot requested a review from rrasouli March 29, 2023 14:27
@openshift-ci-robot
Copy link

/retest-required

Remaining retests: 0 against base HEAD 874a0c6 and 2 for PR HEAD 603a234 in total

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 29, 2023
@aravindhp
Copy link
Contributor

/unhold
/retest-required

@openshift-ci openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 4, 2023
@aravindhp
Copy link
Contributor

aws-e2e-upgrade failed with:

 === RUN   TestWMCO/destroy/Deletion
W0329 21:05:26.425569   14873 warnings.go:70] would violate PodSecurity "restricted:latest": runAsNonRoot != true (pod or container "sleep" must set securityContext.runAsNonRoot=true)
    delete_test.go:126: 
        	Error Trace:	/go/src/github.com/openshift/windows-machine-config-operator/test/e2e/delete_test.go:126
        	            				/go/src/github.com/openshift/windows-machine-config-operator/test/e2e/delete_test.go:31
        	Error:      	Received unexpected error:
        	            	timed out waiting for daemonset noop-ds to scale, current status: {CurrentNumberScheduled:2 NumberMisscheduled:0 DesiredNumberScheduled:2 NumberReady:1 ObservedGeneration:1 UpdatedNumberScheduled:2 NumberAvailable:1 NumberUnavailable:1 CollisionCount:<nil> Conditions:[]}
        	            	github.com/openshift/windows-machine-config-operator/test/e2e.(*testContext).waitUntilDaemonsetScaled
        	            		/go/src/github.com/openshift/windows-machine-config-operator/test/e2e/delete_test.go:239
        	            	github.com/openshift/windows-machine-config-operator/test/e2e.testWindowsNodeDeletion
        	            		/go/src/github.com/openshift/windows-machine-config-operator/test/e2e/delete_test.go:125
        	            	github.com/openshift/windows-machine-config-operator/test/e2e.deletionTestSuite.func1
        	            		/go/src/github.com/openshift/windows-machine-config-operator/test/e2e/delete_test.go:31
        	            	testing.tRunner
        	            		/usr/lib/golang/src/testing/testing.go:1446
        	            	runtime.goexit
        	            		/usr/lib/golang/src/runtime/asm_amd64.s:1594
        	Test:       	TestWMCO/destroy/Deletion
        	Messages:   	error waiting for DaemonSet pods to become ready 

For now we are seeing the error intermittently

@openshift-ci-robot
Copy link

/retest-required

Remaining retests: 0 against base HEAD c6f2937 and 1 for PR HEAD 603a234 in total

@openshift-ci-robot
Copy link

/retest-required

Remaining retests: 0 against base HEAD a15b127 and 0 for PR HEAD 603a234 in total

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Apr 7, 2023

@openshift-cherrypick-robot: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@openshift-merge-robot openshift-merge-robot merged commit 727e665 into openshift:release-4.12 Apr 7, 2023
@openshift-ci-robot
Copy link

@openshift-cherrypick-robot: Jira Issue OCPBUGS-10933: All pull requests linked via external trackers have merged:

Jira Issue OCPBUGS-10933 has been moved to the MODIFIED state.

In response to this:

This is an automated cherry-pick of #1511

/assign mansikulkarni96

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants