Skip to content

OCPBUGS-56115: Understand and eliminate noisy logs for MCN #5136

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

OCPBUGS-56115: Understand and eliminate noisy logs for MCN #5136

wants to merge 2 commits into from

Conversation

naseerahkani
Copy link
Contributor

- What I did
Rearranged the way the errors are checked for the account for the MCN not existing when the node is first created.

- How to verify it

  1. Launch a 4.20 single-node openshift cluster on Clusterbot (note it doesn't need to use aws) launch 4.20 aws,single-node
  2. Enable tech preview (clusterbot options are bit limited which is why, unless something has changed recently, you cannot launch a tech preview single node cluster from the start)
  3. Look at the machine-config-daemon logs oc logs -n openshift-machine-config-operator -c machine-config-daemon machine-config-daemon- -f
  • note that you can get the mcd pod uuid from oc get pods -n openshift-machine-config-operator

- Description for the changelog
Fixes logs that start with "error getting existing MCN" from the upgrade monitor. This should fix bug #56115

@openshift-ci openshift-ci bot requested review from pablintino and umohnani8 June 23, 2025 20:09
@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Jun 23, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: naseerahkani
Once this PR has been reviewed and has the lgtm label, please assign djoshy for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@naseerahkani naseerahkani changed the title fixing noisy logs for MCN OCPBUGS 56115: Understand and eliminate noisy logs for MCN Jun 24, 2025
isabella-janssen
isabella-janssen reviewed Jun 24, 2025
View reviewed changes
pkg/upgrademonitor/upgrade_monitor.go
@@ -403,11 +403,16 @@ func GenerateAndApplyMachineConfigNodeSpec(fgAccessor featuregates.FeatureGateAc
// createOrGetMachineConfigNode gets the named MCN or returns a boolean indicating we need to create one
func createOrGetMachineConfigNode(mcfgClient mcfgclientset.Interface, node *corev1.Node) (*mcfgv1.MachineConfigNode, bool) {
mcNode, err := mcfgClient.MachineconfigurationV1().MachineConfigNodes().Get(context.TODO(), node.Name, metav1.GetOptions{})
if mcNode.Name == "" || (err != nil && apierrors.IsNotFound(err)) {
if err != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: Did you remove the mcNode.Name == "" part of the conditional because it is handled in the apierrors.IsNotFound((err)) case?

@isabella-janssen
Copy link
Member

/payload-job periodic-ci-openshift-release-master-nightly-4.20-e2e-aws-ovn-single-node

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Jun 24, 2025

@isabella-janssen: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.20-e2e-aws-ovn-single-node

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/583b9160-5114-11f0-9df4-6f98e5b226d8-0

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Jun 24, 2025

@naseerahkani: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-azure-ovn-upgrade-out-of-change 8c3f747 link false /test e2e-azure-ovn-upgrade-out-of-change
ci/prow/e2e-gcp-op 8c3f747 link true /test e2e-gcp-op
ci/prow/e2e-hypershift 8c3f747 link true /test e2e-hypershift
ci/prow/e2e-aws-ovn-windows 8c3f747 link false /test e2e-aws-ovn-windows

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@isabella-janssen
Copy link
Member

/jira refresh

@openshift-ci-robot
Copy link
Contributor

@isabella-janssen: No Jira issue is referenced in the title of this pull request.
To reference a jira issue, add 'XYZ-NNN:' to the title of this pull request and request another refresh with /jira refresh.

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@isabella-janssen
Copy link
Member

/retitle OCPBUGS-56115: Understand and eliminate noisy logs for MCN

@openshift-ci openshift-ci bot changed the title OCPBUGS 56115: Understand and eliminate noisy logs for MCN OCPBUGS-56115: Understand and eliminate noisy logs for MCN Jun 24, 2025
@openshift-ci-robot openshift-ci-robot added jira/severity-low Referenced Jira bug's severity is low for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jun 24, 2025
@openshift-ci-robot
Copy link
Contributor

@naseerahkani: This pull request references Jira Issue OCPBUGS-56115, which is invalid:

  • expected the bug to target the "4.20.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

- What I did
Rearranged the way the errors are checked for the account for the MCN not existing when the node is first created.

- How to verify it

  1. Launch a 4.20 single-node openshift cluster on Clusterbot (note it doesn't need to use aws) launch 4.20 aws,single-node
  2. Enable tech preview (clusterbot options are bit limited which is why, unless something has changed recently, you cannot launch a tech preview single node cluster from the start)
  3. Look at the machine-config-daemon logs oc logs -n openshift-machine-config-operator -c machine-config-daemon machine-config-daemon- -f
  • note that you can get the mcd pod uuid from oc get pods -n openshift-machine-config-operator

- Description for the changelog
Fixes logs that start with "error getting existing MCN" from the upgrade monitor. This should fix bug #56115

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@isabella-janssen
Copy link
Member

/payload-aggregate periodic-ci-openshift-release-master-nightly-4.20-e2e-aws-ovn-single-node 5

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Jun 24, 2025

@isabella-janssen: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.20-e2e-aws-ovn-single-node

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/b592d1e0-5135-11f0-8079-89fe6b48309d-0

@isabella-janssen
Copy link
Member

/payload-aggregate periodic-ci-openshift-release-master-nightly-4.20-e2e-aws-ovn-single-node-serial 5

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Jun 24, 2025

@isabella-janssen: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.20-e2e-aws-ovn-single-node-serial

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/bfa789f0-5135-11f0-81b3-d1ffe1ddde27-0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@isabella-janssen isabella-janssen isabella-janssen left review comments

@pablintino pablintino Awaiting requested review from pablintino

@umohnani8 umohnani8 Awaiting requested review from umohnani8

Assignees
No one assigned
Labels
jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. jira/severity-low Referenced Jira bug's severity is low for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@naseerahkani @isabella-janssen @openshift-ci-robot