multiarch: move nightly-4.15 s390x conformance to VPN UPI workflow#80030
multiarch: move nightly-4.15 s390x conformance to VPN UPI workflow#80030ibm-adarsh wants to merge 10 commits into
Conversation
Point ocp-e2e-ovn-remote-libvirt-s390x at libvirt-s390x-vpn with intranet, USE_EXTERNAL_DNS, and openshift-e2e-libvirt-vpn to validate UPI on OCP 4.15.
openshift-ci
Bot
added
the
do-not-merge/work-in-progress
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughSwitches the ocp-e2e-ovn-remote-libvirt-s390x job to VPN settings and adds a qemu_img wrapper plus optional OPENSHIFT_CLIENT_VERSION_OVERRIDE to the UPI libvirt install step. Changess390x Test VPN Infrastructure Migration & UPI oc override
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Suggested labels: Suggested reviewers:
🚥 Pre-merge checks | ✅ 14 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (14 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
openshift-ci
Bot
added
the
needs-ok-to-test
|
Hi @ibm-adarsh. Thanks for your PR. I'm waiting for a openshift member to verify that this patch is reasonable to test. If it is, they should reply with Tip We noticed you've done this a few times! Consider joining the org to skip this step and gain Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Pin the periodic job to build09 for intranet/libvirt-s390x-vpn access.
|
/pj-rehearse periodic-ci-openshift-multiarch-main-nightly-4.15-ocp-e2e-ovn-remote-libvirt-s390x |
|
@ibm-adarsh: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
@ibm-adarsh: needs-ok-to-test label found, no rehearsals will be run |
|
/ok-to-test |
openshift-ci
Bot
added
ok-to-test
|
/pj-rehearse periodic-ci-openshift-multiarch-main-nightly-4.15-ocp-e2e-ovn-remote-libvirt-s390x |
|
@ibm-adarsh: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
The ocp/4.15 libvirt-installer image lacks oc, which upi-install-libvirt needs for release extract. Honor OPENSHIFT_CLIENT_VERSION_OVERRIDE in the step and set fast-4.15 on the nightly-4.15 s390x VPN conformance job.
|
/pj-rehearse periodic-ci-openshift-multiarch-main-nightly-4.15-ocp-e2e-ovn-remote-libvirt-s390x |
|
@ibm-adarsh: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@ci-operator/step-registry/upi/install/libvirt/upi-install-libvirt-commands.sh`:
- Around line 63-64: The curl download of the OpenShift client is currently a
single-shot call and should be hardened: replace the plain curl line that writes
/tmp/openshift-client-linux.tar.gz (the command invoking curl -o ... -L
"https://...${OPENSHIFT_CLIENT_VERSION_OVERRIDE}/$(uname -m | sed
's/aarch64/arm64/;s/x86_64/amd64/;')/openshift-client-linux.tar.gz") with a
retry/timeout loop (or curl --retry, --retry-delay, --max-time flags) that
retries on transient failures and fails after a bounded number of attempts, and
then check the download exit status before running tar; also ensure tar
extraction (tar -xzvf /tmp/openshift-client-linux.tar.gz -C /tmp/bin oc)
validates success and the oc binary is present and executable (chmod u+x
/tmp/bin/oc) to avoid proceeding on partial or corrupted downloads.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Enterprise
Run ID: ee3d917c-aa5e-4ca7-b916-a954d4b313f6
⛔ Files ignored due to path filters (1)
ci-operator/jobs/openshift/multiarch/openshift-multiarch-main-periodics.yamlis excluded by!ci-operator/jobs/**
📒 Files selected for processing (3)
ci-operator/config/openshift/multiarch/openshift-multiarch-main__nightly-4.15.yamlci-operator/step-registry/upi/install/libvirt/upi-install-libvirt-commands.shci-operator/step-registry/upi/install/libvirt/upi-install-libvirt-ref.yaml
🚧 Files skipped from review as they are similar to previous changes (1)
- ci-operator/config/openshift/multiarch/openshift-multiarch-main__nightly-4.15.yaml
The ocp/4.15 libvirt-installer image lacks qemu-img, which is required to resize RHCOS volumes before upload. Install it when OPENSHIFT_CLIENT_VERSION_OVERRIDE is set.
|
/pj-rehearse periodic-ci-openshift-multiarch-main-nightly-4.15-ocp-e2e-ovn-remote-libvirt-s390x |
|
@ibm-adarsh: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
dnf install fails in the CI step container without root. Add a qemu_img helper that uses the host binary when present, otherwise sudo dnf or podman with ocp/4.16:libvirt-installer.
|
/pj-rehearse periodic-ci-openshift-multiarch-main-nightly-4.15-ocp-e2e-ovn-remote-libvirt-s390x |
|
@ibm-adarsh: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
sudo and podman are unavailable in the legacy 4.15 step container. Extract qemu-img from ocp/4.16:libvirt-installer using oc after the client is bootstrapped via OPENSHIFT_CLIENT_VERSION_OVERRIDE.
|
/pj-rehearse periodic-ci-openshift-multiarch-main-nightly-4.15-ocp-e2e-ovn-remote-libvirt-s390x |
|
@ibm-adarsh: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@ci-operator/step-registry/upi/install/libvirt/upi-install-libvirt-commands.sh`:
- Line 66: Replace the hard-coded --filter-by-os="linux/amd64" with a
platform-derived value: detect the host architecture (e.g., via uname -m), map
common outputs to Go/OCI arch names (x86_64 -> amd64, aarch64 -> arm64, etc.),
set a variable like PLATFORM or QEMU_FILTER_OS to "linux/<mapped_arch>", and use
that variable in the qemu-img extraction command's --filter-by-os argument so
the extracted binary matches the executor architecture.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Enterprise
Run ID: 898ff8ab-a079-4e95-b3d3-25995ccfb718
📒 Files selected for processing (1)
ci-operator/step-registry/upi/install/libvirt/upi-install-libvirt-commands.sh
oc image extract expects a directory destination for --path mappings, not a file path. Extract to /tmp/qemu-img-extract and copy qemu-img into /tmp/bin.
|
/pj-rehearse periodic-ci-openshift-multiarch-main-nightly-4.15-ocp-e2e-ovn-remote-libvirt-s390x |
|
@ibm-adarsh: requesting more than one rehearsal in one comment is not supported. If you would like to rehearse multiple specific jobs, please separate the job names by a space in a single command. |
2 similar comments
|
@ibm-adarsh: requesting more than one rehearsal in one comment is not supported. If you would like to rehearse multiple specific jobs, please separate the job names by a space in a single command. |
|
@ibm-adarsh: requesting more than one rehearsal in one comment is not supported. If you would like to rehearse multiple specific jobs, please separate the job names by a space in a single command. |
|
pj-rehearse periodic-ci-openshift-multiarch-main-nightly-4.15-ocp-e2e-ovn-remote-libvirt-s390x periodic-ci-openshift-multiarch-main-nightly-4.15-ocp-e2e-serial-ovn-remote-libvirt-s390x periodic-ci-openshift-multiarch-main-nightly-4.15-ocp-fips-ovn-remote-libvirt-s390x periodic-ci-openshift-multiarch-main-nightly-4.15-ocp-image-ecosystem-ovn-remote-libvirt-s390x periodic-ci-openshift-multiarch-main-nightly-4.15-ocp-jenkins-e2e-ovn-remote-libvirt-s390x periodic-ci-openshift-multiarch-main-nightly-4.15-upgrade-from-nightly-4.14-ocp-ovn-remote-libvirt-s390x periodic-ci-openshift-multiarch-main-nightly-4.16-upgrade-from-nightly-4.15-ocp-ovn-remote-libvirt-s390x |
|
/pj-rehearse periodic-ci-openshift-multiarch-main-nightly-4.15-ocp-e2e-ovn-remote-libvirt-s390x periodic-ci-openshift-multiarch-main-nightly-4.15-ocp-e2e-serial-ovn-remote-libvirt-s390x periodic-ci-openshift-multiarch-main-nightly-4.15-ocp-fips-ovn-remote-libvirt-s390x periodic-ci-openshift-multiarch-main-nightly-4.15-ocp-image-ecosystem-ovn-remote-libvirt-s390x periodic-ci-openshift-multiarch-main-nightly-4.15-ocp-jenkins-e2e-ovn-remote-libvirt-s390x periodic-ci-openshift-multiarch-main-nightly-4.15-upgrade-from-nightly-4.14-ocp-ovn-remote-libvirt-s390x periodic-ci-openshift-multiarch-main-nightly-4.16-upgrade-from-nightly-4.15-ocp-ovn-remote-libvirt-s390x |
|
@ibm-adarsh: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse periodic-ci-openshift-multiarch-main-nightly-4.15-ocp-e2e-serial-ovn-remote-libvirt-s390x |
|
@ibm-adarsh: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse periodic-ci-openshift-multiarch-main-nightly-4.15-ocp-fips-ovn-remote-libvirt-s390x |
|
@ibm-adarsh: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse periodic-ci-openshift-multiarch-main-nightly-4.15-ocp-image-ecosystem-ovn-remote-libvirt-s390x |
|
@ibm-adarsh: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse periodic-ci-openshift-multiarch-main-nightly-4.15-ocp-e2e-ovn-remote-libvirt-s390x |
|
@ibm-adarsh: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse periodic-ci-openshift-multiarch-main-nightly-4.15-ocp-jenkins-e2e-ovn-remote-libvirt-s390x |
|
@ibm-adarsh: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse periodic-ci-openshift-multiarch-main-nightly-4.15-upgrade-from-nightly-4.14-ocp-ovn-remote-libvirt-s390x |
|
@ibm-adarsh: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse periodic-ci-openshift-multiarch-main-nightly-4.16-upgrade-from-nightly-4.15-ocp-ovn-remote-libvirt-s390x |
|
@ibm-adarsh: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse periodic-ci-openshift-multiarch-main-nightly-4.15-upgrade-from-nightly-4.14-ocp-ovn-remote-libvirt-s390x |
|
@ibm-adarsh: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
release:initial resolves to amd64 nightly payloads, which breaks UPI install on s390x libvirt. Restore s390x-initial/s390x-latest for the 4.14->4.15 and 4.15->4.16 upgrade periodics.
|
/pj-rehearse periodic-ci-openshift-multiarch-main-nightly-4.15-upgrade-from-nightly-4.14-ocp-ovn-remote-libvirt-s390x |
|
@ibm-adarsh: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse periodic-ci-openshift-multiarch-main-nightly-4.16-upgrade-from-nightly-4.15-ocp-ovn-remote-libvirt-s390x |
|
@ibm-adarsh: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
[REHEARSALNOTIFIER]
A total of 1091 jobs have been affected by this change. The above listing is non-exhaustive and limited to 25 jobs. A full list of affected jobs can be found here Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
|
@ibm-adarsh: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
2 participants
Summary
Migrate the OpenShift 4.15 s390x multiarch conformance periodic
ocp-e2e-ovn-remote-libvirt-s390xfrom the legacy IPI/libvirt-bastion workflow to the VPN-based UPI workflow already used by newer releases (4.16+), allowing 4.15 s390x nightly payloads to be validated on the IBM Z VPN (orange zone) libvirt infrastructure.Motivation
OpenShift 4.16+ s390x conformance jobs already use:
openshift-e2e-libvirt-vpnlibvirt-s390x-vpnintranetUSE_EXTERNAL_DNS=trueOpenShift 4.15 was still using:
openshift-e2e-libvirt(IPI/terraform via bastion)libvirt-s390x-2sshd-bastionThis no longer matches the current VPN-based IBM Z CI infrastructure.
Aligning 4.15 with the VPN UPI model allows it to use the same connectivity pattern and infrastructure as supported releases without maintaining a separate bastion-only path for an EOL stream.
What Changed
1. Test Definition (
Setting | Before | After -- | -- | -- Workflow | openshift-e2e-libvirt (IPI) | openshift-e2e-libvirt-vpn (UPI) Cluster Profile | libvirt-s390x-2 | libvirt-s390x-vpn Capability | sshd-bastion | intranet Release Dependency | OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE | OPENSHIFT_INSTALL_TARGET Environment | NODE_TUNING, TEST_TYPE | NODE_TUNING, TEST_TYPE, ETCD_DISK_SPEED=slow, USE_EXTERNAL_DNS=trueopenshift-multiarch-main__nightly-4.15.yaml)Why Option A Was Chosen
4.16+ already contain the required tooling.
Avoids maintaining parallel install workflows.
Keeps a single
upi-install-libvirtimplementation.Makes image intent explicit (
libvirt-installer-uipi= image used by UPI installation).Restricts version-specific behavior to the 4.15 override only.
If reviewers prefer minimizing configuration churn, Option B can still be implemented later by reverting the alias additions and introducing a dedicated 4.15 workflow.
Explicitly Not Changed
This PR does not change:
Other 4.15 multiarch libvirt jobs.
IPI-based workflows outside the targeted conformance job.
Heterogeneous jobs.
Image ecosystem jobs.
The OpenShift version under test.
The nightly payload stream under test.
Step-registry installation logic beyond changing the image reference.
The final implementation contains no runtime installation hacks or tooling bootstrapping.
Test Plan
Run rehearsal:
Validate:
Install step uses
libvirt-installer-uipi(4.16 image).No
oc: command not foundfailures.No
qemu-imgextraction/runtime dependency failures.Pre-install and install phases complete successfully.
E2E test execution starts and runs normally.
After successful rehearsal:
If adding explicit build-farm placement:
after adding:
to the ci-operator configuration.
https://prow.ci.openshift.org/job-history/gs/test-platform-results/pr-logs/directory/rehearse-80030-periodic-ci-openshift-multiarch-main-nightly-4.15-ocp-e2e-ovn-remote-libvirt-s390x
Summary by CodeRabbit
This PR updates OpenShift CI configuration in the openshift/release repository to run the nightly OCP 4.15 s390x conformance job using a VPN UPI workflow, and to make the UPI libvirt install step more robust when the legacy 4.15 step container lacks sudo/podman/qemu-img.
What changed in practice
multiarch nightly job (openshift-multiarch-main__nightly-4.15):
UPI libvirt install step:
Why this matters
Files with notable edits