Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OCPBUGS-8502: Fix node scaling issue #343

Merged
merged 1 commit into from Apr 24, 2023
Merged

OCPBUGS-8502: Fix node scaling issue #343

merged 1 commit into from Apr 24, 2023

Conversation

Vincent056
Copy link
Contributor

This PR removes nodestatus when a node is being removed from the cluster, and the status was successful. It also addresses the fail nodestatus issue when adding a new node to the cluster.

OCPBUGS-8502

@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 12, 2023
@openshift-ci openshift-ci bot requested review from jhrozek and xiaojiey April 12, 2023 22:28
@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 12, 2023
@openshift-merge-robot openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 12, 2023
@Vincent056 Vincent056 force-pushed the scale_node branch 3 times, most recently from ac7fddb to d2e69f3 Compare April 13, 2023 01:48
@rhmdnd rhmdnd self-requested a review April 13, 2023 02:51
jhrozek
jhrozek reviewed Apr 13, 2023
View reviewed changes
pkg/controller/status/status_controller.go Outdated
if err := r.client.Delete(context.TODO(), &nodeStatus); err != nil {
return v1alpha1.PhaseError, err
}
continue
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this continue is not needed, right? Because there's no further action in the loop

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we can remove it here

@jhrozek
Copy link
Contributor

jhrozek commented Apr 14, 2023

/retest

@Vincent056
Copy link
Contributor Author

/retest

@Vincent056
Fix Node scaling issue
b03b0e9 
This PR removes nodestatus when a node is being removed from the cluster, and the status was succeed. It also addresses the fail nodestatus issue when adding a new node to the cluster.
@xiaojiey
Copy link

/hold for test

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 19, 2023
@xiaojiey
Copy link

verification pass with 4.13.0-0.nightly-2023-04-18-005127 + code in the PR:

  1. deploy FIO
  2. Create fileintegrity:
    $ oc apply -f -<<EOF
    apiVersion: fileintegrity.openshift.io/v1alpha1
    kind: FileIntegrity
    metadata:
    name: example-fileintegrity
    spec:
    config:
    gracePeriod: 90
    maxBackups: 5
    debug: true
    EOF
  3. Reinit:
    $ oc annotate fileintegrities/example-fileintegrity file-integrity.openshift.io/re-init=
    fileintegrity.fileintegrity.openshift.io/example-fileintegrity annotated
  4. Trigger scaleup
    $ oc scale --replicas=2 machineset xiyuan19-1-jwz6h-worker-us-east-2a -n openshift-machine-api
    machineset.machine.openshift.io/xiyuan19-1-jwz6h-worker-us-east-2a scaled
    $ oc get machineset -n openshift-machine-api
    NAME DESIRED CURRENT READY AVAILABLE AGE
    xiyuan19-1-jwz6h-worker-us-east-2a 2 2 1 1 14h
    xiyuan19-1-jwz6h-worker-us-east-2b 1 1 1 1 14h
    xiyuan19-1-jwz6h-worker-us-east-2c 1 1 1 1 14h
  5. Check the aide pod will be created for the newly added node, the fileintegritynodestatus will show Succeeded
  6. Trigger scale down:
    $ oc get machineset -n openshift-machine-api
    NAME DESIRED CURRENT READY AVAILABLE AGE
    xiyuan19-1-jwz6h-worker-us-east-2a 2 2 2 2 14h
    xiyuan19-1-jwz6h-worker-us-east-2b 1 1 1 1 14h
    xiyuan19-1-jwz6h-worker-us-east-2c 1 1 1 1 14h
    $ oc scale --replicas=0 machineset xiyuan19-1-jwz6h-worker-us-east-2b -n openshift-machine-api
    $ oc get machineset -n openshift-machine-api -w
    NAME DESIRED CURRENT READY AVAILABLE AGE
    xiyuan19-1-jwz6h-worker-us-east-2a 2 2 2 2 14h
    xiyuan19-1-jwz6h-worker-us-east-2b 0 0 14h
    xiyuan19-1-jwz6h-worker-us-east-2c 1 1 1 1 14h
  7. Check the aide pod will be deleted for the deleted node, and fileintegritynodestatus will be deleted:

@xiaojiey
Copy link

/unhold

@openshift-ci openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 19, 2023
@xiaojiey
Copy link

/label qe-approved

@openshift-ci openshift-ci bot added the qe-approved Signifies that QE has signed off on this PR label Apr 19, 2023
jhrozek
jhrozek approved these changes Apr 24, 2023
View reviewed changes
Copy link
Contributor

@jhrozek jhrozek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@jhrozek jhrozek added the docs-approved Signifies that Docs has signed off on this PR label Apr 24, 2023
@jhrozek
Copy link
Contributor

jhrozek commented Apr 24, 2023

added doc ack - this is a bugfix and will be documented with errata

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Apr 24, 2023
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Apr 24, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jhrozek, Vincent056

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@jhrozek jhrozek changed the title OCPBUGS 8502: Fix node scaling issue OCPBUGS-8502: Fix node scaling issue Apr 24, 2023
@openshift-ci-robot openshift-ci-robot added jira/severity-moderate Referenced Jira bug's severity is moderate for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Apr 24, 2023
@openshift-ci-robot
Copy link

@Vincent056: This pull request references Jira Issue OCPBUGS-8502, which is invalid:

  • expected the bug to target the "4.14.0" version, but no target version was set
  • expected the bug to be in one of the following states: NEW, ASSIGNED, POST, but it is MODIFIED instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

This PR removes nodestatus when a node is being removed from the cluster, and the status was successful. It also addresses the fail nodestatus issue when adding a new node to the cluster.

OCPBUGS-8502

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@jhrozek
Copy link
Contributor

jhrozek commented Apr 24, 2023

/jira refresh

@openshift-ci-robot
Copy link

@jhrozek: This pull request references Jira Issue OCPBUGS-8502, which is invalid:

  • expected the bug to target the "4.14.0" version, but no target version was set
  • expected the bug to be in one of the following states: NEW, ASSIGNED, POST, but it is MODIFIED instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@jhrozek
Copy link
Contributor

jhrozek commented Apr 24, 2023

/jira refresh

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Apr 24, 2023
@openshift-ci-robot
Copy link

@jhrozek: This pull request references Jira Issue OCPBUGS-8502, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.14.0) matches configured target version for branch (4.14.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @xiaojiey

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@jhrozek jhrozek merged commit 15f2076 into openshift:master Apr 24, 2023
9 checks passed
@openshift-ci-robot
Copy link

@Vincent056: Jira Issue OCPBUGS-8502: All pull requests linked via external trackers have merged:

Jira Issue OCPBUGS-8502 has been moved to the MODIFIED state.

In response to this:

This PR removes nodestatus when a node is being removed from the cluster, and the status was successful. It also addresses the fail nodestatus issue when adding a new node to the cluster.

OCPBUGS-8502

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Apr 24, 2023

@Vincent056: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-bundle-aws-upgrade b03b0e9 link unknown /test e2e-bundle-aws-upgrade
ci/prow/e2e-bundle-aws b03b0e9 link unknown /test e2e-bundle-aws

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jhrozek jhrozek jhrozek approved these changes

@xiaojiey xiaojiey Awaiting requested review from xiaojiey

@rhmdnd rhmdnd Awaiting requested review from rhmdnd

Assignees

@jhrozek jhrozek

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. docs-approved Signifies that Docs has signed off on this PR jira/severity-moderate Referenced Jira bug's severity is moderate for the branch this PR is targeting. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. qe-approved Signifies that QE has signed off on this PR
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@Vincent056 @jhrozek @xiaojiey @openshift-ci-robot @openshift-merge-robot