Skip to content

platform-external: added e2e upgrade for pext-aws#80100

Open
redhat-chai-bot wants to merge 2 commits into
openshift:mainfrom
redhat-chai-bot:06041435-platform-external-upgrade
Open

platform-external: added e2e upgrade for pext-aws#80100
redhat-chai-bot wants to merge 2 commits into
openshift:mainfrom
redhat-chai-bot:06041435-platform-external-upgrade

Conversation

@redhat-chai-bot

@redhat-chai-bot redhat-chai-bot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Summary

Introduce periodic e2e upgrade workflows for platform type External on AWS, covering both regular (None) and CCM variants.

Changes

Adds 4 new periodic jobs (monthly) to validate OCP upgrades on platform-external AWS installations:

  • e2e-platform-external-aws-upgrade — regular variant (4.21→4.22 and 4.22→5.0)
  • e2e-platform-external-aws-ccm-upgrade — CCM variant (4.21→4.22 and 4.22→5.0)

These jobs reuse the existing openshift-e2e-external-aws workflow with TEST_TYPE=upgrade-conformance to perform cluster upgrades followed by conformance tests.

The CCM variant deploys a standalone aws-cloud-controller-manager during installation and validates that the cluster upgrades successfully with the CCM running outside CVO management.

All jobs report failures to #forum-ocp-splat-alerts-aws.

Summary by CodeRabbit

  • Chores
    • Added monthly upgrade test jobs for external AWS platform upgrades
    • Configured monitoring and alerting infrastructure for new upgrade test jobs
    • Added Cloud Controller Manager variant for external AWS upgrade testing

@openshift-ci openshift-ci Bot requested a review from mtulio June 4, 2026 14:49
@openshift-ci openshift-ci Bot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Jun 4, 2026
@openshift-ci

openshift-ci Bot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Hi @redhat-chai-bot. Thanks for your PR.

I'm waiting for a openshift member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Tip

We noticed you've done this a few times! Consider joining the org to skip this step and gain /lgtm and other bot rights. We recommend asking approvers on your previous PRs to sponsor you.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@coderabbitai

coderabbitai Bot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 40564610-f269-46dd-a37b-ce6c6f3ab87d

📥 Commits

Reviewing files that changed from the base of the PR and between 4f824dc and 9f9eebd.

📒 Files selected for processing (2)
  • ci-operator/config/openshift/release/openshift-release-main__nightly-4.22-upgrade-from-stable-4.21.yaml
  • ci-operator/config/openshift/release/openshift-release-main__nightly-5.0-upgrade-from-stable-4.22.yaml
💤 Files with no reviewable changes (2)
  • ci-operator/config/openshift/release/openshift-release-main__nightly-5.0-upgrade-from-stable-4.22.yaml
  • ci-operator/config/openshift/release/openshift-release-main__nightly-4.22-upgrade-from-stable-4.21.yaml

Walkthrough

This pull request adds two new upgrade test job definitions across two OpenShift release pipeline configurations. The changes introduce e2e-platform-external-aws-upgrade and e2e-platform-external-aws-ccm-upgrade periodic tests for the 4.22-to-4.21 and 5.0-to-4.22 upgrade paths, each configured with monthly scheduling, AWS cluster profiles, Slack-based failure reporting, and optional Cloud Credential Operator CCM enablement.

Changes

AWS Platform External Upgrade Test Configuration

Layer / File(s) Summary
Test definitions for AWS platform external upgrades
ci-operator/config/openshift/release/openshift-release-main__nightly-4.22-upgrade-from-stable-4.21.yaml, ci-operator/config/openshift/release/openshift-release-main__nightly-5.0-upgrade-from-stable-4.22.yaml
Two periodic test entries (e2e-platform-external-aws-upgrade and e2e-platform-external-aws-ccm-upgrade) added to each release file with monthly cron scheduling, reporter configuration targeting #forum-ocp-splat-alerts-aws, openshift-org-aws cluster profile, and optional PLATFORM_EXTERNAL_CCM_ENABLED environment variable in the CCM variant.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~5 minutes

Possibly related PRs

  • openshift/release#80086: Updates report_template in Slack reporter configuration for periodic CI jobs, directly related to the reporter_config patterns added in this PR.

Suggested labels

rehearsals-ack, lifecycle/active

Suggested reviewers

  • mtulio
  • stbenjam
  • petr-muller
🚥 Pre-merge checks | ✅ 15
✅ Passed checks (15 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'platform-external: added e2e upgrade for pext-aws' clearly summarizes the main change: adding e2e upgrade test jobs for platform-external AWS configurations.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed PR modifies only YAML CI/CD configuration files, not Ginkgo test files. Check applies to Go test code with Ginkgo test definitions (It/Describe/Context/When), which are not present in this PR.
Test Structure And Quality ✅ Passed PR contains YAML CI/CD configuration files only, not Ginkgo test code. Custom check for Ginkgo test structure is not applicable.
Microshift Test Compatibility ✅ Passed PR only adds CI job configuration (YAML) for platform-external upgrade tests; no new Ginkgo test code is added, so MicroShift compatibility check does not apply.
Single Node Openshift (Sno) Test Compatibility ✅ Passed PR only modifies YAML CI configuration to add periodic job definitions; no new Ginkgo e2e test code is introduced, so SNO compatibility check does not apply.
Topology-Aware Scheduling Compatibility ✅ Passed PR adds CI operator test configuration only; no deployment manifests, operator code, or scheduling constraints. Check is not applicable.
Ote Binary Stdout Contract ✅ Passed PR modifies only YAML CI configuration files; OTE Binary Stdout Contract check applies to executable code, not configuration files. Check not applicable.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed PR adds CI configuration only (YAML job definitions), not new Ginkgo test code. Custom check for IPv6/disconnected network compatibility in e2e tests is not applicable.
No-Weak-Crypto ✅ Passed PR adds only YAML CI-operator configuration for periodic E2E upgrade test jobs; no cryptographic code, weak crypto algorithms, or insecure secret comparisons are introduced.
Container-Privileges ✅ Passed PR modifies CI-operator config files (not container manifests) defining periodic upgrade jobs. No privileged settings found.
No-Sensitive-Data-In-Logs ✅ Passed PR adds YAML test job configs with non-sensitive environment variables and Slack reporters logging only public job metadata (name, state, URL). No passwords, tokens, keys, or PII found.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci

openshift-ci Bot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: redhat-chai-bot
Once this PR has been reviewed and has the lgtm label, please assign wking for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@mtulio

mtulio commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

/pj-rehearse periodic-ci-openshift-release-main-ci-4.22-upgrade-from-stable-4.21-e2e-external-aws-ccm-upgrade periodic-ci-openshift-release-main-ci-4.22-upgrade-from-stable-4.21-e2e-external-aws-upgrade

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@mtulio: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@mtulio: needs-ok-to-test label found, no rehearsals will be run

@mtulio

mtulio commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

/ok-to-test

@openshift-ci openshift-ci Bot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jun 4, 2026
@mtulio

mtulio commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

/hold
/pj-rehearse periodic-ci-openshift-release-main-ci-4.22-upgrade-from-stable-4.21-e2e-external-aws-ccm-upgrade

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@mtulio: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-ci openshift-ci Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 4, 2026
@mtulio

mtulio commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

I've canceled the job by mistake

/pj-rehearse periodic-ci-openshift-release-main-ci-4.22-upgrade-from-stable-4.21-e2e-external-aws-ccm-upgra

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@mtulio: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@mtulio: job(s): periodic-ci-openshift-release-main-ci-4.22-upgrade-from-stable-4.21-e2e-external-aws-ccm-upgra either don't exist or were not found to be affected, and cannot be rehearsed

@mtulio

mtulio commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

/pj-rehearse periodic-ci-openshift-release-main-ci-4.22-upgrade-from-stable-4.21-e2e-external-aws-ccm-upgrade periodic-ci-openshift-release-main-ci-4.22-upgrade-from-stable-4.21-e2e-external-aws-upgrade

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@mtulio: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@mtulio

mtulio commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

/pj-rehearse periodic-ci-openshift-release-main-ci-4.22-upgrade-from-stable-4.21-e2e-external-aws-upgrade

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@mtulio: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@mtulio

mtulio commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

/pj-rehearse periodic-ci-openshift-release-main-ci-4.20-upgrade-from-stable-4.19-e2e-external-aws-upgrade periodic-ci-openshift-release-main-ci-4.21-upgrade-from-stable-4.20-e2e-external-aws-upgrade

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@mtulio: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@mtulio

mtulio commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

/pj-rehearse periodic-ci-openshift-release-main-ci-4.20-upgrade-from-stable-4.19-e2e-external-aws-ccm-upgrade periodic-ci-openshift-release-main-ci-4.21-upgrade-from-stable-4.20-e2e-external-aws-ccm-upgrade

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@mtulio: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

steps:
cluster_profile: vsphere-elastic
workflow: openshift-upgrade-vsphere-runc
- as: e2e-external-aws-upgrade

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- as: e2e-external-aws-upgrade
- as: e2e-platform-external-aws-upgrade

@mtulio mtulio left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The following review has been submited on Slack:

  • move 4.22 (upgrade from 4.21) to nightly periodic
  • add periodics (for those two jobs) on 5.0 upgrade from 4.22 for nightly builds too
  • remove 4.20 and 4.21 changes from PR https://github.com/openshift/release/pull/80100/changes
  • set all jobs to run monthly
  • preserve notifications
  • rename jobs to the new standard of platform external, replacing external  to platform-external. eg e2e-external-aws-upgrade  must be e2e-platform-external-aws-upgrade

Add monthly periodic upgrade jobs for platform external AWS (regular
and CCM variants) to nightly 4.22 and 5.0 upgrade configs:

- e2e-platform-external-aws-upgrade (4.21->4.22, 4.22->5.0)
- e2e-platform-external-aws-ccm-upgrade (4.21->4.22, 4.22->5.0)

Both jobs use the openshift-e2e-external-aws workflow with
TEST_TYPE=upgrade-conformance, run monthly (0 0 1 * *), and
report failures to #forum-ocp-splat-alerts-aws.
@redhat-chai-bot redhat-chai-bot force-pushed the 06041435-platform-external-upgrade branch from 4f824dc to b108e8a Compare June 8, 2026 18:35
@redhat-chai-bot redhat-chai-bot changed the title Spike/platform-external: introduce e2e upgrade workflows platform-external: added e2e upgrade for pext-aws Jun 8, 2026
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

[REHEARSALNOTIFIER]
@redhat-chai-bot: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name Repo Type Reason
periodic-ci-openshift-release-main-nightly-5.0-upgrade-from-stable-4.22-e2e-platform-external-aws-ccm-upgrade N/A periodic Periodic changed
periodic-ci-openshift-release-main-nightly-4.22-upgrade-from-stable-4.21-e2e-platform-external-aws-ccm-upgrade N/A periodic Periodic changed
periodic-ci-openshift-release-main-nightly-5.0-upgrade-from-stable-4.22-e2e-platform-external-aws-upgrade N/A periodic Periodic changed
periodic-ci-openshift-release-main-nightly-4.22-upgrade-from-stable-4.21-e2e-platform-external-aws-upgrade N/A periodic Periodic changed
Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@mtulio

mtulio commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

/pj-rehearse periodic-ci-openshift-release-main-nightly-4.22-upgrade-from-stable-4.21-e2e-platform-external-aws-upgrade periodic-ci-openshift-release-main-nightly-5.0-upgrade-from-stable-4.22-e2e-platform-external-aws-ccm-upgrade

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@mtulio: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-ci

openshift-ci Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

@redhat-chai-bot: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/rehearse/periodic-ci-openshift-release-main-ci-4.21-upgrade-from-stable-4.20-e2e-external-aws-upgrade 4f824dc link unknown /pj-rehearse periodic-ci-openshift-release-main-ci-4.21-upgrade-from-stable-4.20-e2e-external-aws-upgrade
ci/rehearse/periodic-ci-openshift-release-main-ci-4.20-upgrade-from-stable-4.19-e2e-external-aws-upgrade 4f824dc link unknown /pj-rehearse periodic-ci-openshift-release-main-ci-4.20-upgrade-from-stable-4.19-e2e-external-aws-upgrade
ci/rehearse/periodic-ci-openshift-release-main-ci-4.20-upgrade-from-stable-4.19-e2e-external-aws-ccm-upgrade 4f824dc link unknown /pj-rehearse periodic-ci-openshift-release-main-ci-4.20-upgrade-from-stable-4.19-e2e-external-aws-ccm-upgrade
ci/rehearse/periodic-ci-openshift-release-main-ci-4.22-upgrade-from-stable-4.21-e2e-external-aws-upgrade 2c7ee89 link unknown /pj-rehearse periodic-ci-openshift-release-main-ci-4.22-upgrade-from-stable-4.21-e2e-external-aws-upgrade
ci/rehearse/periodic-ci-openshift-release-main-ci-4.21-upgrade-from-stable-4.20-e2e-external-aws-ccm-upgrade 4f824dc link unknown /pj-rehearse periodic-ci-openshift-release-main-ci-4.21-upgrade-from-stable-4.20-e2e-external-aws-ccm-upgrade
ci/rehearse/periodic-ci-openshift-release-main-ci-4.22-upgrade-from-stable-4.21-e2e-external-aws-ccm-upgrade 2c7ee89 link unknown /pj-rehearse periodic-ci-openshift-release-main-ci-4.22-upgrade-from-stable-4.21-e2e-external-aws-ccm-upgrade
ci/rehearse/periodic-ci-openshift-release-main-nightly-5.0-upgrade-from-stable-4.22-e2e-platform-external-aws-ccm-upgrade 9f9eebd link unknown /pj-rehearse periodic-ci-openshift-release-main-nightly-5.0-upgrade-from-stable-4.22-e2e-platform-external-aws-ccm-upgrade
ci/rehearse/periodic-ci-openshift-release-main-nightly-4.22-upgrade-from-stable-4.21-e2e-platform-external-aws-upgrade 9f9eebd link unknown /pj-rehearse periodic-ci-openshift-release-main-nightly-4.22-upgrade-from-stable-4.21-e2e-platform-external-aws-upgrade

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. ok-to-test Indicates a non-member PR verified by an org member that is safe to test.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants