Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OCPBUGS-33046: improve status reporting for pinned image sets #4339

Merged
merged 1 commit into from
May 7, 2024

Conversation

hexfusion
Copy link
Contributor

@hexfusion hexfusion commented May 1, 2024

This PR improves status updates to MCN and MCP while PinnedImageSets are reconciling.

Changes:

  • node controller status properly calculates update machines
  • node controller will sync if a pool attached to a node is currently reconciling pinnedImageSets. This gives a faster feedback loop when nodes have updated or errored.
  • add poolSyncronizer logic to MCN status to provide finer grained status reporting for MCP poolSyncronizers.
  • refactoring of status population in PinnedImageSet Manager
  • additional feature gating

@openshift-ci-robot openshift-ci-robot added jira/severity-moderate Referenced Jira bug's severity is moderate for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. labels May 1, 2024
@openshift-ci-robot
Copy link
Contributor

@hexfusion: This pull request references Jira Issue OCPBUGS-33046, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.16.0) matches configured target version for branch (4.16.0)
  • bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @sergiordlr

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 1, 2024
@hexfusion
Copy link
Contributor Author

/hold testing

@openshift-ci openshift-ci bot requested a review from sergiordlr May 1, 2024 01:40
@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 1, 2024
@openshift-ci openshift-ci bot requested review from cdoern and djoshy May 1, 2024 01:40
@openshift-merge-robot openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 1, 2024
@openshift-ci-robot
Copy link
Contributor

@hexfusion: This pull request references Jira Issue OCPBUGS-33046, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.16.0) matches configured target version for branch (4.16.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @sergiordlr

In response to this:

This PR improves status updates to MCN and MCP while PinnedImageSets are reconciling.

Changes:

  • node controller status properly calculates update machines
  • node controller will sync if a pool attached to a node is currently reconciling pinnedImageSets. This gives a faster feedback loop when nodes have updated or errored.
  • add poolSyncronizer logic to MCN status to provide finer grained status reporting for MCP poolSyncronizers.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@sergiordlr
Copy link

sergiordlr commented May 3, 2024

Verified using these steps:

  1. Enable techpreview
  2. Create a pinnedimageset to pin a single image
apiVersion: machineconfiguration.openshift.io/v1alpha1
kind: PinnedImageSet
metadata:
  labels:
    machineconfiguration.openshift.io/role: worker
  name: my-worker-pinned-images
spec:
  pinnedImages:
  - name: "quay.io/.../...@sha256:0415f56ccc05526f2af5a7ae8654baec97d4a6....."

3.Add 10 nodes to the worker pool
4. Check that the pinnedset poolSynchronizersStatus is updated accordingly and fast enough

oc get mcp worker -o yaml 
...
  poolSynchronizersStatus:
  - availableMachineCount: 12
    machineCount: 12
    poolSynchronizerType: PinnedImageSets
    readyMachineCount: 12
    unavailableMachineCount: 0
    updatedMachineCount: 12

It took a few seconds to update the status with the right values.

We add the qe-approved label

/label qe-approved

@openshift-ci openshift-ci bot added the qe-approved Signifies that QE has signed off on this PR label May 3, 2024
Copy link
Contributor

@sinnykumari sinnykumari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks sane.

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 3, 2024
@hexfusion
Copy link
Contributor Author

/hold cancel

@openshift-ci openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 3, 2024
@@ -180,19 +185,16 @@ func generateAndApplyMachineConfigNodes(
}
newC.DeepCopyInto(&newMCNode.Status.Conditions[i])
} else if newChildCondition != nil && condition.Type == newChildCondition.Type {
childDNEOrIsTheSame = false
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the value of childDNEOrIsTheSame could only ever be changed here but then could never be consumed.

Copy link
Contributor

@yuqi-zhang yuqi-zhang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

Just a question inline (but non-blocking)

pkg/daemon/pinned_image_set.go Outdated Show resolved Hide resolved
@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label May 7, 2024
@openshift-ci-robot
Copy link
Contributor

/retest-required

Remaining retests: 0 against base HEAD 2766d7c and 2 for PR HEAD bede30f in total

Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
@openshift-ci openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label May 7, 2024
@yuqi-zhang
Copy link
Contributor

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label May 7, 2024
Copy link
Contributor

openshift-ci bot commented May 7, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hexfusion, sinnykumari, yuqi-zhang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [sinnykumari,yuqi-zhang]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot
Copy link
Contributor

/retest-required

Remaining retests: 0 against base HEAD 5a6e8b8 and 2 for PR HEAD 8cbdc2a in total

@hexfusion
Copy link
Contributor Author

/retest-required

Copy link
Contributor

openshift-ci bot commented May 7, 2024

@hexfusion: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/security 8cbdc2a link false /test security

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@openshift-merge-bot openshift-merge-bot bot merged commit 7dea103 into openshift:master May 7, 2024
14 of 15 checks passed
@openshift-ci-robot
Copy link
Contributor

@hexfusion: Jira Issue OCPBUGS-33046: All pull requests linked via external trackers have merged:

Jira Issue OCPBUGS-33046 has been moved to the MODIFIED state.

In response to this:

This PR improves status updates to MCN and MCP while PinnedImageSets are reconciling.

Changes:

  • node controller status properly calculates update machines
  • node controller will sync if a pool attached to a node is currently reconciling pinnedImageSets. This gives a faster feedback loop when nodes have updated or errored.
  • add poolSyncronizer logic to MCN status to provide finer grained status reporting for MCP poolSyncronizers.
  • refactoring of status population in PinnedImageSet Manager
  • additional feature gating

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@hexfusion hexfusion deleted the mcn-status branch May 7, 2024 20:28
@openshift-bot
Copy link
Contributor

[ART PR BUILD NOTIFIER]

This PR has been included in build ose-machine-config-operator-container-v4.17.0-202405072356.p0.g7dea103.assembly.stream.el9 for distgit ose-machine-config-operator.
All builds following this will include this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/severity-moderate Referenced Jira bug's severity is moderate for the branch this PR is targeting. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. qe-approved Signifies that QE has signed off on this PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants