OCPBUGS-27948: Tuned rendering and MCP detection improvements #928

MarSik · 2024-01-25T09:14:16Z

This is a backport of three related PRs: #878, #833 and #924 that improve the rendering mode during the installation bootstrap phase.

The most notable change is the added assumption that basic master and worker MachineConfigPools with default labels will always exist even though they are not yet available in the input manifest folder.

render-bootcmd-mc can only compute the kernel args for a MachineConfigPool that matches the node it is executed on, so we add the MachineConfigPool name as input parameter to filter the applicable PerformanceProfiles. Signed-off-by: Jose Luis Ojosnegros Manchón <jojosneg@redhat.com>

* feat: update render command to create default MCPs added default MCP creation to help correctly render out the resources for performance profile to use Signed-off-by: ehila <ehila@redhat.com> upkeep: fix spelling Signed-off-by: ehila <ehila@redhat.com> * feat: add default mcpools to tuned renderer Signed-off-by: ehila <ehila@redhat.com> --------- Signed-off-by: ehila <ehila@redhat.com>

openshift-ci · 2024-01-25T09:14:32Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: MarSik

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [MarSik]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot · 2024-01-25T09:16:26Z

@MarSik: This pull request references Jira Issue OCPBUGS-27948, which is invalid:

expected dependent Jira Issue OCPBUGS-22095 to target a version in 4.16.0, but it targets "4.15.0" instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

MarSik · 2024-01-25T09:17:27Z

/jira refresh

openshift-ci-robot · 2024-01-25T09:17:34Z

@MarSik: This pull request references Jira Issue OCPBUGS-27948, which is valid. The bug has been moved to the POST state.

6 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.15.0) matches configured target version for branch (4.15.0)
bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)
dependent bug Jira Issue OCPBUGS-22095 is in the state ON_QA, which is one of the valid states (MODIFIED, ON_QA, VERIFIED)
dependent Jira Issue OCPBUGS-22095 targets the "4.16.0" version, which is one of the valid target versions: 4.16.0
bug has dependents

Requesting review from QA contact:
/cc @shajmakh

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

MarSik · 2024-01-25T09:18:17Z

@jlojosnegros @vitus133 @eggfoobar Took a me a while to untangle the patch ordering, but this should be it.

openshift-ci-robot · 2024-01-25T09:23:52Z

@MarSik: This pull request references Jira Issue OCPBUGS-27948, which is valid.

6 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.15.0) matches configured target version for branch (4.15.0)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)
dependent bug Jira Issue OCPBUGS-22095 is in the state ON_QA, which is one of the valid states (MODIFIED, ON_QA, VERIFIED)
dependent Jira Issue OCPBUGS-22095 targets the "4.16.0" version, which is one of the valid target versions: 4.16.0
bug has dependents

Requesting review from QA contact:
/cc @shajmakh

In response to this:

This is a backport of two PRs: #878 and #833 that improve the rendering mode during the installation bootstrap phase.

The most notable change is the added assumption that basic master and worker MachineConfigPools with default labels will always exist even though they are not yet available in the input manifest folder.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

MarSik · 2024-01-25T10:12:09Z

/label backport-risk-assessed

ffromani

LGTM

a bit too clever, but somehow self-testing. I guess there's no simple way to test before to commit.

ffromani · 2024-01-25T11:35:00Z

hack/render-sync.sh

-done
-rm -r "${ARTIFACT_DIR}"
+
+function join_by { local IFS="$1"; shift; echo "$*"; }


this is clever! My gut feeling is this is a bit too clever, but I don't have strong arguments against.

I did not invent this one actually. But it uses bash features cleverly indeed.

MarSik · 2024-01-25T14:00:54Z

I guess there's no simple way to test before to commit.

Actually there is a way. I executed the code and it generated the exact same manifests we already had in the repo. git reported no changes. I also executed the e2e local tests that validate the rendering and all passed. That probably means the output is still valid and matches the rendered yamls.

MarSik · 2024-01-26T10:07:56Z

/retest-required

jlojosnegros · 2024-01-30T15:51:08Z

pkg/tuned/cmd/render/render.go

+		return fmt.Errorf("Unable to get PerformanceProfile to apply using MachineConfigPool %s. error : %w", mcpName, err)
+	}
+
+	if len(filteredPerformanceProfiles) == 0 {


We had just changed this behaviour in master with #924 so the absence of a performance profile is not considered an error.

https://github.com/openshift/cluster-node-tuning-operator/pull/924/files#diff-f3655cb37779a7069442b7a63925fe91be42f16cafb43968dba654f44abf19c5

I added that patch here too! Thanks for noticing that.

This command should render a MC from the available sources. PerformanceProfile is a source, but it is not mandatory. So, as there are scenarios where this command should work with no additional PP let's the command proceed even if there is no PP Signed-off-by: Jose Luis Ojosnegros Manchón <jojosneg@redhat.com>

openshift-ci-robot · 2024-02-01T08:28:12Z

@MarSik: This pull request references Jira Issue OCPBUGS-27948, which is valid.

6 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.15.0) matches configured target version for branch (4.15.0)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)
dependent bug Jira Issue OCPBUGS-22095 is in the state ON_QA, which is one of the valid states (MODIFIED, ON_QA, VERIFIED)
dependent Jira Issue OCPBUGS-22095 targets the "4.16.0" version, which is one of the valid target versions: 4.16.0
bug has dependents

Requesting review from QA contact:
/cc @shajmakh

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

This is a backport of three related PRs: #878, #833 and #924 that improve the rendering mode during the installation bootstrap phase.

The most notable change is the added assumption that basic master and worker MachineConfigPools with default labels will always exist even though they are not yet available in the input manifest folder.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

yanirq · 2024-02-01T13:41:53Z

/retest

yanirq · 2024-02-04T19:24:25Z

/retest

shajmakh · 2024-02-05T07:55:20Z

/label cherry-pick-approved

yanirq · 2024-02-05T09:25:36Z

/retest-required

yanirq · 2024-02-05T12:54:11Z

/lgtm

openshift-ci-robot · 2024-02-05T13:29:32Z

/retest-required

Remaining retests: 0 against base HEAD b4be046 and 2 for PR HEAD 644c518 in total

openshift-ci · 2024-02-05T15:12:23Z

@MarSik: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

openshift-ci-robot · 2024-02-05T15:15:53Z

@MarSik: Jira Issue OCPBUGS-27948: All pull requests linked via external trackers have merged:

openshift/cluster-node-tuning-operator#928

Jira Issue OCPBUGS-27948 has been moved to the MODIFIED state.

In response to this:

This is a backport of three related PRs: #878, #833 and #924 that improve the rendering mode during the installation bootstrap phase.

The most notable change is the added assumption that basic master and worker MachineConfigPools with default labels will always exist even though they are not yet available in the input manifest folder.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-bot · 2024-02-05T16:12:32Z

[ART PR BUILD NOTIFIER]

This PR has been included in build cluster-node-tuning-operator-container-v4.15.0-202402051538.p0.ga8d42a7.assembly.stream for distgit cluster-node-tuning-operator.
All builds following this will include this PR.

openshift-merge-robot · 2024-02-05T23:07:51Z

Fix included in accepted release 4.15.0-0.nightly-2024-02-05-163411

yanirq · 2024-02-06T11:57:39Z

/cherry-pick release-4.14

openshift-cherrypick-robot · 2024-02-06T11:58:32Z

@yanirq: #928 failed to apply on top of branch "release-4.14":

Applying: NTO: Add MCP name to filter PP (#878)
Using index info to reconstruct a base tree...
A	pkg/tuned/cmd/render/cmd.go
A	pkg/tuned/cmd/render/render.go
Falling back to patching base and 3-way merge...
CONFLICT (modify/delete): pkg/tuned/cmd/render/render.go deleted in HEAD and modified in NTO: Add MCP name to filter PP (#878). Version NTO: Add MCP name to filter PP (#878) of pkg/tuned/cmd/render/render.go left in tree.
Auto-merging pkg/performanceprofile/cmd/render/cmd.go
CONFLICT (content): Merge conflict in pkg/performanceprofile/cmd/render/cmd.go
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 NTO: Add MCP name to filter PP (#878)
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

In response to this:

/cherry-pick release-4.14

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-merge-robot · 2024-02-14T05:30:12Z

Fix included in accepted release 4.15.0-0.nightly-2024-02-12-213938

…ift#928) * NTO: Add MCP name to filter PP (openshift#878) render-bootcmd-mc can only compute the kernel args for a MachineConfigPool that matches the node it is executed on, so we add the MachineConfigPool name as input parameter to filter the applicable PerformanceProfiles. Signed-off-by: Jose Luis Ojosnegros Manchón <jojosneg@redhat.com> * OCPBUGS-22095: Add default MCP objects for rendering (openshift#833) * feat: update render command to create default MCPs added default MCP creation to help correctly render out the resources for performance profile to use Signed-off-by: ehila <ehila@redhat.com> upkeep: fix spelling Signed-off-by: ehila <ehila@redhat.com> * feat: add default mcpools to tuned renderer Signed-off-by: ehila <ehila@redhat.com> --------- Signed-off-by: ehila <ehila@redhat.com> * Enhance render sync to include bootstrap rendering tests * Render MC without additional PP This command should render a MC from the available sources. PerformanceProfile is a source, but it is not mandatory. So, as there are scenarios where this command should work with no additional PP let's the command proceed even if there is no PP Signed-off-by: Jose Luis Ojosnegros Manchón <jojosneg@redhat.com> * Render sync --------- Signed-off-by: Jose Luis Ojosnegros Manchón <jojosneg@redhat.com> Signed-off-by: ehila <ehila@redhat.com> Co-authored-by: Jose Luis Ojosnegros <jojosneg@redhat.com> Co-authored-by: E Hila <ehila@redhat.com>

jlojosnegros and others added 2 commits January 25, 2024 10:08

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 25, 2024

openshift-ci bot requested review from ffromani and jlojosnegros January 25, 2024 09:14

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 25, 2024

MarSik changed the title ~~WIP: Tuned rendering and MCP detection improvements~~ OCPBUGS-27948: WIP: Tuned rendering and MCP detection improvements Jan 25, 2024

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 25, 2024

openshift-ci-robot added jira/severity-moderate Referenced Jira bug's severity is moderate for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. labels Jan 25, 2024

openshift-ci-robot added the jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. label Jan 25, 2024

openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jan 25, 2024

openshift-ci bot requested a review from shajmakh January 25, 2024 09:17

Enhance render sync to include bootstrap rendering tests

204f4d8

openshift-ci bot added the backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. label Jan 25, 2024

ffromani reviewed Jan 25, 2024

View reviewed changes

MarSik changed the title ~~OCPBUGS-27948: WIP: Tuned rendering and MCP detection improvements~~ OCPBUGS-27948: Tuned rendering and MCP detection improvements Jan 29, 2024

jlojosnegros reviewed Jan 30, 2024

View reviewed changes

jlojosnegros and others added 2 commits February 1, 2024 09:26

Render sync

644c518

MarSik force-pushed the 415-rendering branch from 64de115 to 644c518 Compare February 1, 2024 08:26

openshift-ci bot added the cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. label Feb 5, 2024

openshift-ci bot assigned yanirq Feb 5, 2024

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Feb 5, 2024

openshift-merge-bot bot merged commit a8d42a7 into openshift:release-4.15 Feb 5, 2024
14 checks passed

openshift-ci-robot mentioned this pull request Feb 28, 2024

[release-4.15] OCPBUGS-27948: Render: Add MCSelector to missing default MCPs #959

Merged

yanirq mentioned this pull request Mar 17, 2024

Backporting Tuned rendering and MCP detection improvements #971

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OCPBUGS-27948: Tuned rendering and MCP detection improvements #928

OCPBUGS-27948: Tuned rendering and MCP detection improvements #928

MarSik commented Jan 25, 2024 •

edited

openshift-ci bot commented Jan 25, 2024

openshift-ci-robot commented Jan 25, 2024

MarSik commented Jan 25, 2024

openshift-ci-robot commented Jan 25, 2024

MarSik commented Jan 25, 2024

openshift-ci-robot commented Jan 25, 2024

MarSik commented Jan 25, 2024

ffromani left a comment

ffromani Jan 25, 2024

MarSik Jan 25, 2024

MarSik commented Jan 25, 2024

MarSik commented Jan 26, 2024

jlojosnegros Jan 30, 2024 •

edited

MarSik Feb 1, 2024

openshift-ci-robot commented Feb 1, 2024

yanirq commented Feb 1, 2024

yanirq commented Feb 4, 2024

shajmakh commented Feb 5, 2024

yanirq commented Feb 5, 2024

yanirq commented Feb 5, 2024

openshift-ci-robot commented Feb 5, 2024

openshift-ci bot commented Feb 5, 2024

openshift-ci-robot commented Feb 5, 2024

openshift-bot commented Feb 5, 2024

openshift-merge-robot commented Feb 5, 2024

yanirq commented Feb 6, 2024

openshift-cherrypick-robot commented Feb 6, 2024

openshift-merge-robot commented Feb 14, 2024

OCPBUGS-27948: Tuned rendering and MCP detection improvements #928

OCPBUGS-27948: Tuned rendering and MCP detection improvements #928

Conversation

MarSik commented Jan 25, 2024 • edited

openshift-ci bot commented Jan 25, 2024

openshift-ci-robot commented Jan 25, 2024

MarSik commented Jan 25, 2024

openshift-ci-robot commented Jan 25, 2024

MarSik commented Jan 25, 2024

openshift-ci-robot commented Jan 25, 2024

MarSik commented Jan 25, 2024

ffromani left a comment

Choose a reason for hiding this comment

ffromani Jan 25, 2024

Choose a reason for hiding this comment

MarSik Jan 25, 2024

Choose a reason for hiding this comment

MarSik commented Jan 25, 2024

MarSik commented Jan 26, 2024

jlojosnegros Jan 30, 2024 • edited

Choose a reason for hiding this comment

MarSik Feb 1, 2024

Choose a reason for hiding this comment

openshift-ci-robot commented Feb 1, 2024

yanirq commented Feb 1, 2024

yanirq commented Feb 4, 2024

shajmakh commented Feb 5, 2024

yanirq commented Feb 5, 2024

yanirq commented Feb 5, 2024

openshift-ci-robot commented Feb 5, 2024

openshift-ci bot commented Feb 5, 2024

openshift-ci-robot commented Feb 5, 2024

openshift-bot commented Feb 5, 2024

openshift-merge-robot commented Feb 5, 2024

yanirq commented Feb 6, 2024

openshift-cherrypick-robot commented Feb 6, 2024

openshift-merge-robot commented Feb 14, 2024

MarSik commented Jan 25, 2024 •

edited

jlojosnegros Jan 30, 2024 •

edited