Skip to content

EXPERIMENT: Revert "OTA-2084: Conditionally deploy console plugin when AgenticRun CRD is present" - #1435

Closed
redhat-chai-bot wants to merge 1 commit into
openshift:mainfrom
redhat-chai-bot:revert-1425-agenticrun-unconditional
Closed

EXPERIMENT: Revert "OTA-2084: Conditionally deploy console plugin when AgenticRun CRD is present"#1435
redhat-chai-bot wants to merge 1 commit into
openshift:mainfrom
redhat-chai-bot:revert-1425-agenticrun-unconditional

Conversation

@redhat-chai-bot

@redhat-chai-bot redhat-chai-bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Summary

This is an experimental revert to test a hypothesis — do not merge.

Reverts PR #1425 (merge commit 027e419) to test whether the agenticrun controller running unconditionally is causing the SNO upgrade pathological events failure in the 5.0 nightly stream.

Context

The 5.0 amd64 nightly stream has been rejected for 3 consecutive payloads. The sole blocking failure is periodic-ci-openshift-release-main-ci-5.0-e2e-aws-ovn-upgrade-single-node-aggregated.

The root cause is the pathological events monitor test detecting machine-api's clusteroperator emitting 25 "Status upgrade Progressing" k8s events during SNO micro upgrades (threshold is 20). This test first started failing consistently (9/10 runs) in payload 5.0.0-0.nightly-2026-07-27-110939.

Why PR #1425 is suspected

PR #1425 removed the shouldEnableAgenticRunController() guard that previously gated the agenticrun controller behind TechPreviewNoUpgrade. After this change, the controller runs on all clusters, adding:

  • CRD discovery checks on every sync
  • Console plugin ensure/disable logic
  • Additional reconciliation queue entries

On SNO, where all control plane workloads share a single node, this extra API server activity during upgrade likely increases the rate at which machine-api's status condition toggles, pushing the k8s event count from below 20 to 25. No other component PR correlates with the failure window. Upgrade timing data confirms the upgrade itself is not slower — the event emission rate increased, not the duration.

Test plan

Run /payload for the failing SNO upgrade job against this revert PR:

Revert details

This reverts commit 027e419.


@smg247 requested via Chai Bot

Summary by CodeRabbit

  • New Features

    • Added release and capability metadata for the console plugin.
    • Standardized the console plugin and backend service identity.
    • Added deployment image resolution during manifest rendering.
  • Behavior Changes

    • Agentic Run processing now runs only for the supported feature set.
    • Updates are queued conditionally based on feature availability.
    • Removed automatic console plugin lifecycle management.

This reverts commit 027e419, reversing
changes made to 55dfed3.
@openshift-ci

openshift-ci Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 28, 2026
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jul 28, 2026
@openshift-ci-robot

openshift-ci-robot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

@redhat-chai-bot: This pull request references OTA-2084 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Summary

This is an experimental revert to test a hypothesis — do not merge.

Reverts PR #1425 (merge commit 027e419) to test whether the agenticrun controller running unconditionally is causing the SNO upgrade pathological events failure in the 5.0 nightly stream.

Context

The 5.0 amd64 nightly stream has been rejected for 3 consecutive payloads. The sole blocking failure is periodic-ci-openshift-release-main-ci-5.0-e2e-aws-ovn-upgrade-single-node-aggregated.

The root cause is the pathological events monitor test detecting machine-api's clusteroperator emitting 25 "Status upgrade Progressing" k8s events during SNO micro upgrades (threshold is 20). This test first started failing consistently (9/10 runs) in payload 5.0.0-0.nightly-2026-07-27-110939.

Why PR #1425 is suspected

PR #1425 removed the shouldEnableAgenticRunController() guard that previously gated the agenticrun controller behind TechPreviewNoUpgrade. After this change, the controller runs on all clusters, adding:

  • CRD discovery checks on every sync
  • Console plugin ensure/disable logic
  • Additional reconciliation queue entries

On SNO, where all control plane workloads share a single node, this extra API server activity during upgrade likely increases the rate at which machine-api's status condition toggles, pushing the k8s event count from below 20 to 25. No other component PR correlates with the failure window. Upgrade timing data confirms the upgrade itself is not slower — the event emission rate increased, not the duration.

Test plan

Run /payload for the failing SNO upgrade job against this revert PR:

Revert details

This reverts commit 027e419.


@smg247 requested via Chai Bot

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci

openshift-ci Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: redhat-chai-bot
Once this PR has been reviewed and has the lgtm label, please assign hongkailiu for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai

coderabbitai Bot commented Jul 28, 2026

Copy link
Copy Markdown

Walkthrough

The change adds release and capability annotations to console plugin manifests, renames the plugin service identity, templates its deployment image, removes console-plugin orchestration from AgenticRun reconciliation, and gates AgenticRun controller startup and queueing on TechPreviewNoUpgrade.

Changes

AgenticRun and console plugin integration

Layer / File(s) Summary
Console plugin manifest metadata
install/0000_50_cluster-update-console-plugin_*.yaml, pkg/payload/render_test.go
Console plugin resources receive release annotations, the deployment uses .Images, and service/plugin references use openshift-cluster-update-console-plugin.
AgenticRun controller simplification
pkg/agenticrun/controller.go, pkg/agenticrun/controller_test.go, pkg/agenticrun/consoleplugin.go, pkg/agenticrun/bindata/*
Console plugin orchestration, embedded asset access, CRD caching, and related controller state are removed from AgenticRun reconciliation.
Feature-gated AgenticRun reconciliation
pkg/cvo/cvo.go, pkg/cvo/availableupdates.go, pkg/cvo/status.go, pkg/cvo/*_test.go
AgenticRun controller startup and queueing occur only when the required feature set is TechPreviewNoUpgrade, with corresponding test setup updates.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Operator
  participant FeatureSetCheck
  participant AgenticRunController
  Operator->>FeatureSetCheck: evaluate requiredFeatureSet
  FeatureSetCheck-->>Operator: enable only for TechPreviewNoUpgrade
  Operator->>AgenticRunController: start worker when enabled
  Operator->>AgenticRunController: enqueue available-update or status-pruning sync
Loading

Possibly related PRs

Suggested reviewers: jhadvig

🚥 Pre-merge checks | ✅ 14 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (14 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately identifies this as an experimental revert of the console-plugin deployment change.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed Modified test files only changed setup/body code; no Ginkgo titles or dynamic test names were added or altered.
Test Structure And Quality ✅ Passed No Ginkgo tests were added; changed tests are table-driven/unit-style, with cleanup on httptest servers and no cluster waits or missing timeouts.
Microshift Test Compatibility ✅ Passed The PR only changes unit tests/manifests; no new Ginkgo e2e specs or MicroShift-unsafe test APIs were added.
Single Node Openshift (Sno) Test Compatibility ✅ Passed No new Ginkgo e2e tests were added; only unit tests/manifests changed, with no multi-node assumptions or SNO-specific issues found.
Topology-Aware Scheduling Compatibility ✅ Passed No topology-sensitive scheduling constraints were introduced; the deployment only uses os: linux nodeSelector, maxUnavailable: 1, and an infra-only toleration.
Ote Binary Stdout Contract ✅ Passed Touched files only add buffered fmt writes/tests; no stdout writes or suite/main setup were added, and main.go already redirects klog to stderr.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed No new Ginkgo e2e tests were added; only unit-test fixtures changed, with no IPv4-only assumptions or external connectivity requirements.
No-Weak-Crypto ✅ Passed Diff adds no MD5/SHA1/DES/RC4/3DES/Blowfish/ECB, custom crypto, or non-constant-time secret compares; only crypto/tls remains.
Container-Privileges ✅ Passed No changed manifest adds privileged/hostPID/hostNetwork/hostIPC/SYS_ADMIN/root escalation; deployment keeps restricted-v3, allowPrivilegeEscalation:false, drop ALL, runAsNonRoot:true.
No-Sensitive-Data-In-Logs ✅ Passed No new logs expose secrets/PII/internal hostnames; the only added log is a generic disabled-controller message, and other log changes are removals.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@redhat-chai-bot

Copy link
Copy Markdown
Contributor Author

/payload 5.0 nightly blocking e2e-aws-ovn-upgrade-single-node

@openshift-ci

openshift-ci Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

@redhat-chai-bot: it appears that you have attempted to use some version of the payload command, but your comment was incorrectly formatted and cannot be acted upon. See the docs for usage info.

@smg247

smg247 commented Jul 28, 2026

Copy link
Copy Markdown
Member

/payload-aggregate periodic-ci-openshift-release-main-ci-5.0-e2e-aws-upgrade-ovn-single-node 10

@openshift-ci

openshift-ci Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

@smg247: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-main-ci-5.0-e2e-aws-upgrade-ovn-single-node

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/02316420-8a89-11f1-9951-2ac80d187a2b-0

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
pkg/cvo/cvo.go (1)

1222-1226: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add coverage for both controller-gate outcomes.

This helper controls startup and both enqueue paths. Add tests for TechPreviewNoUpgrade and disabled sets such as Default and DevPreviewNoUpgrade, including assertions that no AgenticRun work is queued when disabled.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/cvo/cvo.go` around lines 1222 - 1226, Add tests for
Operator.shouldEnableAgenticRunController covering TechPreviewNoUpgrade as
enabled and Default and DevPreviewNoUpgrade as disabled. Verify the disabled
cases also prevent AgenticRun work from being queued through both enqueue paths
that depend on this gate.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@pkg/cvo/cvo.go`:
- Around line 1222-1226: Add tests for Operator.shouldEnableAgenticRunController
covering TechPreviewNoUpgrade as enabled and Default and DevPreviewNoUpgrade as
disabled. Verify the disabled cases also prevent AgenticRun work from being
queued through both enqueue paths that depend on this gate.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: 28a718c6-7faf-4a31-9d97-b3e05a7fd555

📥 Commits

Reviewing files that changed from the base of the PR and between 72f34cb and 02387cd.

📒 Files selected for processing (17)
  • install/0000_50_cluster-update-console-plugin_10_namespace.yaml
  • install/0000_50_cluster-update-console-plugin_15_serviceaccount.yaml
  • install/0000_50_cluster-update-console-plugin_20_networkpolicy.yaml
  • install/0000_50_cluster-update-console-plugin_30_configmap.yaml
  • install/0000_50_cluster-update-console-plugin_50_deployment.yaml
  • install/0000_50_cluster-update-console-plugin_60_service.yaml
  • install/0000_50_cluster-update-console-plugin_90_consoleplugin.yaml
  • pkg/agenticrun/bindata/assets.go
  • pkg/agenticrun/bindata/assets/networkpolicy-allow-console.yaml
  • pkg/agenticrun/consoleplugin.go
  • pkg/agenticrun/controller.go
  • pkg/agenticrun/controller_test.go
  • pkg/cvo/availableupdates.go
  • pkg/cvo/availableupdates_test.go
  • pkg/cvo/cvo.go
  • pkg/cvo/status.go
  • pkg/payload/render_test.go
💤 Files with no reviewable changes (6)
  • pkg/agenticrun/bindata/assets/networkpolicy-allow-console.yaml
  • pkg/agenticrun/consoleplugin.go
  • pkg/agenticrun/bindata/assets.go
  • pkg/cvo/availableupdates_test.go
  • pkg/agenticrun/controller.go
  • pkg/agenticrun/controller_test.go

@redhat-chai-bot

Copy link
Copy Markdown
Contributor Author

/payload-aggregate-with-prs periodic-ci-openshift-release-main-ci-5.0-e2e-aws-upgrade-ovn-single-node 10 openshift/console-operator#1200

@openshift-ci

openshift-ci Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

@redhat-chai-bot: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-main-ci-5.0-e2e-aws-upgrade-ovn-single-node

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/f0afade0-8aa8-11f1-8113-af31792b0819-0

@smg247

smg247 commented Jul 29, 2026

Copy link
Copy Markdown
Member

/close

@openshift-ci openshift-ci Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 29, 2026
@openshift-ci

openshift-ci Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci openshift-ci Bot closed this Jul 29, 2026
@openshift-ci

openshift-ci Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

@smg247: Closed this PR.

Details

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants