Add vSphere disconnected Windows Containers periodic jobs for 4.18-4.22#78004
Closed
rrasouli wants to merge 1 commit into
Closed
Add vSphere disconnected Windows Containers periodic jobs for 4.18-4.22#78004rrasouli wants to merge 1 commit into
rrasouli wants to merge 1 commit into
Conversation
Contributor
WalkthroughAdds CI jobs and step-registry workflows to run Windows Containers (WINC) on disconnected vSphere+OVN clusters: new presubmits/periodics/scheduled and optional debug tests (4.18–4.22), a provision/deprovision workflow and chains, WMCO Konflux disconnected mirroring, enhanced diagnostics, and OWNERS/metadata updates. Changes
Sequence Diagram(s)sequenceDiagram
participant Trigger as Developer / Scheduler
participant Prow as Prow (rehearse/periodic/presubmit)
participant CI as ci-operator
participant Registry as Bastion Mirror Registry
participant Prov as Provision Chain
participant Cluster as Disconnected vSphere Cluster
participant Tests as WINC e2e Tests
participant Deprov as Deprovision Chain
Trigger->>Prow: schedule / trigger job
Prow->>CI: start ci-operator job (target workflow)
CI->>Registry: prepare mirror (WMCO index & related images)
CI->>Prov: invoke provision chain (bastion, OVN, Windows setup)
Prov->>Cluster: provision disconnected vSphere cluster
CI->>Cluster: apply ImageDigest/TagMirrorSets, CatalogSource
CI->>Tests: run openshift-extended-test workflow
Tests-->>CI: test results
CI->>Deprov: invoke deprovision chain
Deprov->>Cluster: teardown cluster & bastion
Estimated code review effort🎯 4 (Complex) | ⏱️ ~40 minutes 🚥 Pre-merge checks | ✅ 9 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (9 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
|
@rrasouli, Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
rrasouli
force-pushed
the
vsphere-disconnected-winc-jobs
branch
from
e2c3d73 to
a464aaa
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 14
🧹 Nitpick comments (6)
ci-operator/step-registry/openshift/windows/setup-wmco-konflux-disconnected/openshift-windows-setup-wmco-konflux-disconnected-ref.metadata.json (1)
3-12: Inconsistent ownership: individual user mixed with group owners.All other new metadata files in this PR (workflow, provision chain, deprovision chain) use only the
winc-approvers/winc-reviewersgroups. Listingrrasoulias an individual approver/reviewer alongside the groups here creates a one-off ownership model for this step and also risks becoming stale if the author's involvement changes. Consider relying on thewinc-approvers/winc-reviewersgroups alone (and adding the individual to those groups if needed) for consistency.Proposed change
"owners": { "approvers": [ - "rrasouli", "winc-approvers" ], "reviewers": [ - "winc-reviewers", - "rrasouli" + "winc-reviewers" ] }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ci-operator/step-registry/openshift/windows/setup-wmco-konflux-disconnected/openshift-windows-setup-wmco-konflux-disconnected-ref.metadata.json` around lines 3 - 12, The owners section mixes an individual ("rrasouli") with group owners; update the "owners" object by removing the individual entries from both "approvers" and "reviewers" so only the groups "winc-approvers" and "winc-reviewers" remain (if that individual needs access, add them to the corresponding group instead of listing them here); modify the entries under the owners -> approvers and owners -> reviewers arrays to remove "rrasouli" and keep "winc-approvers" / "winc-reviewers" only.ci-operator/step-registry/openshift/windows/setup-wmco-konflux-disconnected/openshift-windows-setup-wmco-konflux-disconnected-commands.sh (4)
104-114: Guard the jq relatedImages traversal against bundles withoutrelatedImages.
jq -r 'select(.schema=="olm.bundle").relatedImages[].image'will error out withCannot iterate over nullon any matched bundle entry that lacks arelatedImagesarray, aborting the whole script (set -euo pipefail). WMCO FBC bundles currently include it, but this is a cheap hardening:♻️ Safer jq expression
- related_images=$(jq -r 'select(.schema=="olm.bundle").relatedImages[].image' "${temp_dir}/catalog.json" | sort -u) + related_images=$(jq -r 'select(.schema=="olm.bundle") | .relatedImages[]? | .image' "${temp_dir}/catalog.json" | sort -u)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ci-operator/step-registry/openshift/windows/setup-wmco-konflux-disconnected/openshift-windows-setup-wmco-konflux-disconnected-commands.sh` around lines 104 - 114, The jq extraction for related_images can fail on bundles without relatedImages; update the related_images assignment that currently uses jq -r 'select(.schema=="olm.bundle").relatedImages[].image' to use a safe traversal (use jq's optional operator and/or default to an empty array so null is not iterated) when reading "${temp_dir}/catalog.json"; ensure the log/error handling around the related_images variable and the existing if [ -z "${related_images}" ] check remain correct.
139-190: Translation logic looks right; consider logging which mirror target was produced per source.The
src_no_digest/img_sha/repo_path_no_registryparsing, theregistry.redhat.io/openshift4-wincw/...→ Konfluxquay.io/redhat-user-workloads/...-release-${version_tag}translation, and the retry/hard-fail on mirroring a required related image all line up with the existing WMCO IDMS convention inwindows-conf-operator-olmv1-commands.sh. Returning non-zero whenimages_mirrored==0(line 187) is a good backstop.One small resiliency note: the per-image retry uses
3attempts here vs.5for the index image (line 73). Given that each related-image failure aborts the entire job, consider aligning on5to absorb transient Konflux/quay throttling.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ci-operator/step-registry/openshift/windows/setup-wmco-konflux-disconnected/openshift-windows-setup-wmco-konflux-disconnected-commands.sh` around lines 139 - 190, The loop that mirrors related images should both log the final mirror target for each source and increase the per-image retry limit to 5 to match the index image resiliency; update the mirroring block around the until oc image mirror call (the loop reading related_images, the retries variable, and the oc image mirror invocation) to log the resolved mirrored_img alongside the source (src_img or img) before attempting the mirror and change the retry condition from 3 to 5 so failures retry up to five times before returning 1 and before pushing idms_sources and incrementing images_mirrored.
49-96: Trap override is intentional but fragile — consolidate cleanup.Line 53 sets a
RETURNtrap cleaningnew_pull_secret; line 95 overrides it to also cleantemp_dir. This works today, but if a future edit adds areturnbetween lines 53 and 95 that only cleansnew_pull_secret, it's easy to miss. Declaring both temp paths upfront and installing the full trap once keeps intent clearer:♻️ Single consolidated trap
- new_pull_secret="$(mktemp)" + new_pull_secret="$(mktemp)" + temp_dir="$(mktemp -d)" + trap 'rm -f "${new_pull_secret:-}"; [[ -n "${temp_dir:-}" ]] && rm -rf "${temp_dir}"' RETURN registry_cred=$(head -n 1 "/var/run/vault/mirror-registry/registry_creds" | base64 -w 0) jq --argjson a "{\"${mirror_registry}\": {\"auth\": \"$registry_cred\"}}" '.auths |= . + $a' "${CLUSTER_PROFILE_DIR}/pull-secret" > "${new_pull_secret}" - trap 'rm -f "${new_pull_secret:-}"' RETURN @@ - log "Extracting catalog.json from FBC index..." - local temp_dir - temp_dir="$(mktemp -d)" - trap 'rm -f "${new_pull_secret:-}"; [[ -n "${temp_dir:-}" ]] && rm -rf "${temp_dir}"' RETURN + log "Extracting catalog.json from FBC index..."🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ci-operator/step-registry/openshift/windows/setup-wmco-konflux-disconnected/openshift-windows-setup-wmco-konflux-disconnected-commands.sh` around lines 49 - 96, The current script sets trap for new_pull_secret early and later overrides it to also remove temp_dir; instead declare temp_dir (the mktemp -d assignment) before any trap is installed, remove the first trap command, and install a single unified trap once that cleans both "${new_pull_secret:-}" and "${temp_dir:-}" on RETURN; update references to the trap and ensure mirrored_index/tmp creation steps use those same variable names (new_pull_secret and temp_dir) so the single trap reliably cleans both.
344-382: CatalogSource readiness check is solid, but consider waiting for the MCO rollout before returning.The patch to
image.config.openshift.io/clusterat line 326 triggers an MCO rollout on all pools so nodes pick up the newadditionalTrustedCA. This function returns as soon as the CatalogSource reportsREADY, but subsequent steps (e.g.,openshift-windows-install-wmco, workload pulls) may race the MCP rollout and hit sporadic x509 failures on the first try.If the downstream chain does not already wait for MCPs to settle, adding a brief
oc wait mcp --all --for=condition=Updated=True --timeout=20mafter the patch would eliminate a whole class of flakes without meaningfully extending runtime (the IDMS apply triggers its own rollout anyway).🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ci-operator/step-registry/openshift/windows/setup-wmco-konflux-disconnected/openshift-windows-setup-wmco-konflux-disconnected-commands.sh` around lines 344 - 382, When CatalogSource reports READY inside the readiness loop (the block that logs "CatalogSource is READY" and currently returns 0), wait for the MachineConfigPool rollout to finish to avoid x509 flakes by invoking oc wait; add a run_command call such as run_command "oc wait mcp --all --for=condition=Updated=True --timeout=20m" (with a brief log before/after) immediately before the existing return 0 in the CatalogSource READY branch so the script blocks until MCPs have settled.ci-operator/step-registry/cucushift/winc/prepare/cucushift-winc-prepare-commands.sh (1)
300-324: Machineset/node readiness observability additions look good.Dumping the machineset YAML up-front, iterating
oc describeon eachwinworkermachine, and printing finaloc get nodes -l kubernetes.io/os=windows -o wideafter the 30-minute wait will make BYOH configuration failures in disconnected runs much easier to diagnose.Minor: the
whileloop at line 304 stillecho -n "."on every 10s tick for up to the machineset provisioning duration with no timeout — consider bounding it similarly to theoc wait nodescall so a stuck machineset is surfaced explicitly rather than consuming the whole job wall-clock.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ci-operator/step-registry/cucushift/winc/prepare/cucushift-winc-prepare-commands.sh` around lines 300 - 324, The machineset readiness while-loop that checks jsonpath {.status.readyReplicas} for winworker_machineset_name/winworker_machineset_replicas currently spins indefinitely; add a timeout to bound it (similar to the oc wait nodes --timeout=30m usage). Modify the loop around the check for readyReplicas (the while using oc -n openshift-machine-api get machineset/${winworker_machineset_name} -o 'jsonpath={.status.readyReplicas}') to either call oc wait with a timeout for the machineset readyReplicas or implement an elapsed/time-left check (e.g., start time + 30m, sleep 10s per tick) and on timeout print a clear error message including winworker_machineset_name and exit non-zero so a stuck machineset fails the job explicitly.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In
`@ci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.18__amd64-nightly.yaml`:
- Around line 2250-2263: The cron schedule for the
vsphere-ipi-disconnected-ovn-winc-f28 job is set to "0 2 15 * *" (runs only on
the 15th); change the cron value under that job's cron key to "0 2 * * *" to
make it run nightly at 02:00 UTC; update the cron field in the same job block
(look for cron: and the workflow name
cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc) and commit the
new schedule.
In
`@ci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.18-periodics.yaml`:
- Around line 40535-40572: The pod spec mounts a volume named "gcs-credentials"
at mountPath "/secrets/gcs" but the "volumes:" list does not contain a volume
with name "gcs-credentials"; add a volume entry named "gcs-credentials" under
the existing volumes block (near the other entries like "boskos" and
"result-aggregator") that references the appropriate secret (e.g., secretName:
gcs-credentials) or configMap as needed, and include any required secret
items/keys so the mount is valid.
In
`@ci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.19-periodics.yaml`:
- Line 41889: The cron in the prowgen output is monthly ("cron: 0 6 15 * *") but
the PR says it should be nightly; open the source config
ci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.19__amd64-nightly.yaml,
locate the test entry vsphere-ipi-disconnected-ovn-winc-f28 and change its cron
from "0 6 15 * *" to "0 6 * * *" (and make the same change for the equivalent
files for releases 4.18, 4.20, 4.21, 4.22), then regenerate the prowgen output;
alternatively, if the schedule was intended to be monthly, update the PR
description to reflect the monthly cadence instead of changing the config.
In
`@ci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.20-periodics.yaml`:
- Around line 44394-44431: The pod spec mounts /secrets/gcs under name
gcs-credentials but the volumes array never defines a volume named
gcs-credentials, causing an invalid Pod and missing GCS secret; add a volume
entry with name: gcs-credentials that sources the correct secret (e.g.,
secretName: gcs-credentials) and include any required items (such as path:
service-account.json) so the mount /secrets/gcs/service-account.json is present
for the ci-operator container referencing
--gcs-upload-secret=/secrets/gcs/service-account.json.
In
`@ci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.21-periodics.yaml`:
- Line 48255: The cron entry for the amd64-nightly jobs is set to run monthly
("0 14 15 * *") instead of nightly; update the cron line in each
__amd64-nightly.yaml so the day-of-month field is "*" (e.g. change "0 14 15 * *"
to "0 14 * * *" for the 4.21 amd64-nightly job) and similarly set the other
releases to their intended nightly hours (4.18 -> "0 2 * * *", 4.19 -> "0 6 * *
*", 4.20 -> "0 10 * * *", 4.22 -> "0 18 * * *"); after making these edits to the
amd64-nightly job entries, re-run prowgen to regenerate the periodics YAML.
In
`@ci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.22-periodics.yaml`:
- Around line 50788-50825: The pod spec references a volumeMount named
"gcs-credentials" but no corresponding volume is defined in the "volumes"
section; add a volume entry named "gcs-credentials" under the existing volumes
block so Kubernetes can mount it. Specifically, add a volume with name:
gcs-credentials and point it to the appropriate secret (e.g., secretName:
gcs-credentials, and include items if specific keys/paths are required) so the
existing mountPath /secrets/gcs resolves correctly; ensure the new entry is
aligned with the other secret-based volumes.
- Line 50738: The cron schedule for the nightly job is wrong: replace the
literal cron value "cron: 0 18 15 * *" with "cron: 0 18 * * *" for the
amd64-nightly job (also update the matching source config entry named
openshift-openshift-tests-private-release-4.22__amd64-nightly.yaml). Also fix
the missing volume definition referenced by the container: locate the container
that has a volumeMount for "gcs-credentials" and either add a corresponding pod
spec volume named "gcs-credentials" (with the appropriate secret/hostPath/GCS
credentials source) or remove the volumeMount if that credential is not
required.
In
`@ci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/provision/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-provision-chain.yaml`:
- Around line 4-10: The chain is invoking ref: ipi-install-install without
running the pre-install RBAC step; insert ref: ipi-install-rbac immediately
before ref: ipi-install-install in the chain so the sequence includes
ipi-install-rbac then ipi-install-install (ensure you update the list that
currently contains ipi-conf-vsphere, vsphere-provision-bastionhost,
mirror-images-payload, ovn-conf,
ovn-conf-hybrid-manifest-with-custom-vxlan-port, ipi-conf-mirror to include
ipi-install-rbac directly before ipi-install-install).
In
`@ci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/winc/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-workflow.yaml`:
- Around line 1-11: Move the workflow and its provision/deprovision chains out
of the incorrect disconnected/winc/ directory into disconnected/ovn/winc/ so
they match the metadata canonical path; specifically relocate the workflow named
"cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc" and the chains
"cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-provision" and
"cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-deprovision"
into disconnected/ovn/winc/, remove the duplicate files left in
disconnected/winc/, and update any references or imports that pointed to the old
location to reference disconnected/ovn/winc/ instead.
In
`@ci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/winc/deprovision/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-deprovision-chain.metadata.json`:
- Around line 1-2: The metadata file
cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-deprovision-chain.metadata.json
is duplicated and points to the chain path
"cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/deprovision/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-deprovision-chain.yaml";
remove the redundant copy under the /winc/deprovision/ location or move the
YAML/metadata pair so there is a single authoritative set under the OVN
location, ensuring only one metadata file references the chain path and deleting
the orphaned duplicate to avoid cross-directory pointers.
In
`@ci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/winc/deprovision/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-deprovision-chain.yaml`:
- Around line 1-8: The chain YAML is located in the wrong registry tree; move
this chain (named
"cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-deprovision")
and its sibling provision/workflow YAMLs into the ovn/winc/deprovision directory
that matches the metadata path so the registry layout aligns with the PR
metadata; keep the chain content and step refs ("gather-mirror-registry",
"cucushift-installer-rehearse-vsphere-ipi-deprovision",
"vsphere-deprovision-bastionhost") unchanged and verify the registry
index/metadata references still point to the new location.
In
`@ci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/winc/provision/OWNERS`:
- Around line 1-4: Delete the misplaced OWNERS file and the OVN-specific
provision chain files from the non-OVN directory: remove OWNERS and the
duplicate
cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-provision-chain.yaml
plus its associated metadata files from
cucushift/installer/rehearse/vsphere/ipi/disconnected/winc/provision/, leaving
the correct copies in disconnected/ovn/winc/provision/; ensure no remaining
references to those deleted files exist in any job or registry entries (search
for
cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-provision-chain.yaml
and remove/update references if found).
In
`@ci-operator/step-registry/openshift/windows/setup-wmco-konflux-disconnected/openshift-windows-setup-wmco-konflux-disconnected-commands.sh`:
- Around line 309-322: The ConfigMap key is hardcoded to port 5000
("${mirror_registry_host}..5000") which breaks non-default ports; update the
block that sets mirror_registry_host/client_ca_cert to also extract the port
from mirror_registry (e.g., mirror_registry_port="$(echo "${mirror_registry}" |
cut -d: -f2)" or via parameter expansion), build the registry key using both
host and port ("${mirror_registry_host}:${mirror_registry_port}" or default to
5000 only if port is empty), and replace the two oc create configmap invocations
(the lines that pass --from-file with "${mirror_registry_host}..5000") to use
the derived host:port key so the ConfigMap matches the actual mirror_registry
value used by IDMS/CatalogSource.
---
Nitpick comments:
In
`@ci-operator/step-registry/cucushift/winc/prepare/cucushift-winc-prepare-commands.sh`:
- Around line 300-324: The machineset readiness while-loop that checks jsonpath
{.status.readyReplicas} for
winworker_machineset_name/winworker_machineset_replicas currently spins
indefinitely; add a timeout to bound it (similar to the oc wait nodes
--timeout=30m usage). Modify the loop around the check for readyReplicas (the
while using oc -n openshift-machine-api get
machineset/${winworker_machineset_name} -o 'jsonpath={.status.readyReplicas}')
to either call oc wait with a timeout for the machineset readyReplicas or
implement an elapsed/time-left check (e.g., start time + 30m, sleep 10s per
tick) and on timeout print a clear error message including
winworker_machineset_name and exit non-zero so a stuck machineset fails the job
explicitly.
In
`@ci-operator/step-registry/openshift/windows/setup-wmco-konflux-disconnected/openshift-windows-setup-wmco-konflux-disconnected-commands.sh`:
- Around line 104-114: The jq extraction for related_images can fail on bundles
without relatedImages; update the related_images assignment that currently uses
jq -r 'select(.schema=="olm.bundle").relatedImages[].image' to use a safe
traversal (use jq's optional operator and/or default to an empty array so null
is not iterated) when reading "${temp_dir}/catalog.json"; ensure the log/error
handling around the related_images variable and the existing if [ -z
"${related_images}" ] check remain correct.
- Around line 139-190: The loop that mirrors related images should both log the
final mirror target for each source and increase the per-image retry limit to 5
to match the index image resiliency; update the mirroring block around the until
oc image mirror call (the loop reading related_images, the retries variable, and
the oc image mirror invocation) to log the resolved mirrored_img alongside the
source (src_img or img) before attempting the mirror and change the retry
condition from 3 to 5 so failures retry up to five times before returning 1 and
before pushing idms_sources and incrementing images_mirrored.
- Around line 49-96: The current script sets trap for new_pull_secret early and
later overrides it to also remove temp_dir; instead declare temp_dir (the mktemp
-d assignment) before any trap is installed, remove the first trap command, and
install a single unified trap once that cleans both "${new_pull_secret:-}" and
"${temp_dir:-}" on RETURN; update references to the trap and ensure
mirrored_index/tmp creation steps use those same variable names (new_pull_secret
and temp_dir) so the single trap reliably cleans both.
- Around line 344-382: When CatalogSource reports READY inside the readiness
loop (the block that logs "CatalogSource is READY" and currently returns 0),
wait for the MachineConfigPool rollout to finish to avoid x509 flakes by
invoking oc wait; add a run_command call such as run_command "oc wait mcp --all
--for=condition=Updated=True --timeout=20m" (with a brief log before/after)
immediately before the existing return 0 in the CatalogSource READY branch so
the script blocks until MCPs have settled.
In
`@ci-operator/step-registry/openshift/windows/setup-wmco-konflux-disconnected/openshift-windows-setup-wmco-konflux-disconnected-ref.metadata.json`:
- Around line 3-12: The owners section mixes an individual ("rrasouli") with
group owners; update the "owners" object by removing the individual entries from
both "approvers" and "reviewers" so only the groups "winc-approvers" and
"winc-reviewers" remain (if that individual needs access, add them to the
corresponding group instead of listing them here); modify the entries under the
owners -> approvers and owners -> reviewers arrays to remove "rrasouli" and keep
"winc-approvers" / "winc-reviewers" only.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 0aaee36d-16c6-449c-95f4-89a6b0d55e2b
📒 Files selected for processing (52)
ci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.18.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.18__amd64-nightly.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.19.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.19__amd64-nightly.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.20.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.20__amd64-nightly.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.21.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.21__amd64-nightly.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.22.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.22__amd64-nightly.yamlci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.18-periodics.yamlci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.18-presubmits.yamlci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.19-periodics.yamlci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.19-presubmits.yamlci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.20-periodics.yamlci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.20-presubmits.yamlci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.21-periodics.yamlci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.21-presubmits.yamlci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.22-periodics.yamlci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.22-presubmits.yamlci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/OWNERSci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/OWNERSci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-workflow.metadata.jsonci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-workflow.yamlci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/deprovision/OWNERSci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/deprovision/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-deprovision-chain.metadata.jsonci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/deprovision/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-deprovision-chain.yamlci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/provision/OWNERSci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/provision/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-provision-chain.metadata.jsonci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/provision/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-provision-chain.yamlci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/winc/OWNERSci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/winc/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-workflow.metadata.jsonci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/winc/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-workflow.yamlci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/winc/deprovision/OWNERSci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/winc/deprovision/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-deprovision-chain.metadata.jsonci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/winc/deprovision/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-deprovision-chain.yamlci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/winc/provision/OWNERSci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/winc/provision/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-provision-chain.metadata.jsonci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/winc/provision/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-provision-chain.yamlci-operator/step-registry/cucushift/winc/prepare/cucushift-winc-prepare-commands.shci-operator/step-registry/openshift/windows/OWNERSci-operator/step-registry/openshift/windows/install-wmco/OWNERSci-operator/step-registry/openshift/windows/install-wmco/openshift-windows-install-wmco-ref.metadata.jsonci-operator/step-registry/openshift/windows/setup-wmco-konflux-disconnected/OWNERSci-operator/step-registry/openshift/windows/setup-wmco-konflux-disconnected/openshift-windows-setup-wmco-konflux-disconnected-commands.shci-operator/step-registry/openshift/windows/setup-wmco-konflux-disconnected/openshift-windows-setup-wmco-konflux-disconnected-ref.metadata.jsonci-operator/step-registry/openshift/windows/setup-wmco-konflux-disconnected/openshift-windows-setup-wmco-konflux-disconnected-ref.yamlci-operator/step-registry/openshift/windows/setup-wmco-konflux/OWNERSci-operator/step-registry/openshift/windows/setup-wmco-konflux/openshift-windows-setup-wmco-konflux-ref.metadata.jsonci-operator/step-registry/openshift/windows/winc-workflow/OWNERSci-operator/step-registry/openshift/windows/winc-workflow/openshift-windows-winc-workflow-chain.metadata.jsonclusters/app.ci/publicize/_config.yaml
💤 Files with no reviewable changes (8)
- ci-operator/step-registry/openshift/windows/OWNERS
- ci-operator/step-registry/openshift/windows/setup-wmco-konflux/OWNERS
- ci-operator/step-registry/openshift/windows/install-wmco/OWNERS
- ci-operator/step-registry/openshift/windows/winc-workflow/OWNERS
- ci-operator/step-registry/openshift/windows/setup-wmco-konflux/openshift-windows-setup-wmco-konflux-ref.metadata.json
- ci-operator/step-registry/openshift/windows/install-wmco/openshift-windows-install-wmco-ref.metadata.json
- ci-operator/step-registry/openshift/windows/winc-workflow/openshift-windows-winc-workflow-chain.metadata.json
- clusters/app.ci/publicize/_config.yaml
| secretName: result-aggregator | ||
| - agent: kubernetes | ||
| cluster: vsphere02 | ||
| cron: 0 2 15 * * |
Contributor
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# Check the file and surrounding context at line 40485
sed -n '40480,40495p' ci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.18-periodics.yamlRepository: openshift/release
Length of output: 477
🏁 Script executed:
# Check the volume definitions around the mentioned lines
sed -n '40535,40575p' ci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.18-periodics.yamlRepository: openshift/release
Length of output: 1330
🏁 Script executed:
# Look at the source config file mentioned
if [ -f ci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.18__amd64-nightly.yaml ]; then
head -100 ci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.18__amd64-nightly.yaml | tail -50
else
find . -name "*release-4.18*amd64-nightly*" -type f | head -5
fiRepository: openshift/release
Length of output: 1003
🏁 Script executed:
# Search for other nightly jobs in the same file to understand the pattern
grep -n "cron:" ci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.18-periodics.yaml | grep "nightly" -B2 -A2 | head -20Repository: openshift/release
Length of output: 43
🏁 Script executed:
# Check more broadly for nightly job patterns
rg "amd64-nightly.*" ci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.18-periodics.yaml -A 5 | head -40Repository: openshift/release
Length of output: 1674
Fix cron schedule (monthly→nightly) and add missing volume definition.
Two issues prevent this job from working correctly:
-
Cron schedule is monthly, not nightly: Line 40485 sets
cron: 0 2 15 * *, which runs only on the 15th of each month. The job name contains "nightly" and should run daily at 02:00 UTC. Change to0 2 * * *. -
Critical: Missing
gcs-credentialsvolume definition. The container mountsgcs-credentialsat/secrets/gcs(line 40535), but the volume is not defined in the pod spec (lines 40551–40572). The pod will fail to start. Add:
- name: gcs-credentials
secret:
secretName: gcs-credentialsAlso update the matching config in ci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.18__amd64-nightly.yaml if it contains the same cron schedule.
Comment on lines
+40535
to
+40572
| - mountPath: /secrets/gcs | ||
| name: gcs-credentials | ||
| readOnly: true | ||
| - mountPath: /usr/local/github-credentials | ||
| name: github-credentials-openshift-ci-robot-private-git-cloner | ||
| readOnly: true | ||
| - mountPath: /secrets/manifest-tool | ||
| name: manifest-tool-local-pusher | ||
| readOnly: true | ||
| - mountPath: /etc/pull-secret | ||
| name: pull-secret | ||
| readOnly: true | ||
| - mountPath: /etc/report | ||
| name: result-aggregator | ||
| readOnly: true | ||
| serviceAccountName: ci-operator | ||
| volumes: | ||
| - name: boskos | ||
| secret: | ||
| items: | ||
| - key: credentials | ||
| path: credentials | ||
| secretName: boskos-credentials | ||
| - name: ci-pull-credentials | ||
| secret: | ||
| secretName: ci-pull-credentials | ||
| - name: github-credentials-openshift-ci-robot-private-git-cloner | ||
| secret: | ||
| secretName: github-credentials-openshift-ci-robot-private-git-cloner | ||
| - name: manifest-tool-local-pusher | ||
| secret: | ||
| secretName: manifest-tool-local-pusher | ||
| - name: pull-secret | ||
| secret: | ||
| secretName: registry-pull-credentials | ||
| - name: result-aggregator | ||
| secret: | ||
| secretName: result-aggregator |
Contributor
There was a problem hiding this comment.
Define the mounted gcs-credentials volume.
Lines 40535-40537 mount gcs-credentials, but the pod volumes list does not define it. Kubernetes will reject/fail this pod spec because the volume mount references a missing volume.
🛠️ Proposed fix
- name: ci-pull-credentials
secret:
secretName: ci-pull-credentials
+ - name: gcs-credentials
+ secret:
+ secretName: gce-sa-credentials-gcs-publisher
- name: github-credentials-openshift-ci-robot-private-git-cloner
secret:
secretName: github-credentials-openshift-ci-robot-private-git-cloner📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| - mountPath: /secrets/gcs | |
| name: gcs-credentials | |
| readOnly: true | |
| - mountPath: /usr/local/github-credentials | |
| name: github-credentials-openshift-ci-robot-private-git-cloner | |
| readOnly: true | |
| - mountPath: /secrets/manifest-tool | |
| name: manifest-tool-local-pusher | |
| readOnly: true | |
| - mountPath: /etc/pull-secret | |
| name: pull-secret | |
| readOnly: true | |
| - mountPath: /etc/report | |
| name: result-aggregator | |
| readOnly: true | |
| serviceAccountName: ci-operator | |
| volumes: | |
| - name: boskos | |
| secret: | |
| items: | |
| - key: credentials | |
| path: credentials | |
| secretName: boskos-credentials | |
| - name: ci-pull-credentials | |
| secret: | |
| secretName: ci-pull-credentials | |
| - name: github-credentials-openshift-ci-robot-private-git-cloner | |
| secret: | |
| secretName: github-credentials-openshift-ci-robot-private-git-cloner | |
| - name: manifest-tool-local-pusher | |
| secret: | |
| secretName: manifest-tool-local-pusher | |
| - name: pull-secret | |
| secret: | |
| secretName: registry-pull-credentials | |
| - name: result-aggregator | |
| secret: | |
| secretName: result-aggregator | |
| - mountPath: /secrets/gcs | |
| name: gcs-credentials | |
| readOnly: true | |
| - mountPath: /usr/local/github-credentials | |
| name: github-credentials-openshift-ci-robot-private-git-cloner | |
| readOnly: true | |
| - mountPath: /secrets/manifest-tool | |
| name: manifest-tool-local-pusher | |
| readOnly: true | |
| - mountPath: /etc/pull-secret | |
| name: pull-secret | |
| readOnly: true | |
| - mountPath: /etc/report | |
| name: result-aggregator | |
| readOnly: true | |
| serviceAccountName: ci-operator | |
| volumes: | |
| - name: boskos | |
| secret: | |
| items: | |
| - key: credentials | |
| path: credentials | |
| secretName: boskos-credentials | |
| - name: ci-pull-credentials | |
| secret: | |
| secretName: ci-pull-credentials | |
| - name: gcs-credentials | |
| secret: | |
| secretName: gce-sa-credentials-gcs-publisher | |
| - name: github-credentials-openshift-ci-robot-private-git-cloner | |
| secret: | |
| secretName: github-credentials-openshift-ci-robot-private-git-cloner | |
| - name: manifest-tool-local-pusher | |
| secret: | |
| secretName: manifest-tool-local-pusher | |
| - name: pull-secret | |
| secret: | |
| secretName: registry-pull-credentials | |
| - name: result-aggregator | |
| secret: | |
| secretName: result-aggregator |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@ci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.18-periodics.yaml`
around lines 40535 - 40572, The pod spec mounts a volume named "gcs-credentials"
at mountPath "/secrets/gcs" but the "volumes:" list does not contain a volume
with name "gcs-credentials"; add a volume entry named "gcs-credentials" under
the existing volumes block (near the other entries like "boskos" and
"result-aggregator") that references the appropriate secret (e.g., secretName:
gcs-credentials) or configMap as needed, and include any required secret
items/keys so the mount is valid.
Comment on lines
+309
to
+322
| # Configure cluster to trust mirror registry CA certificate | ||
| log "Configuring cluster to trust mirror registry CA certificate..." | ||
| local client_ca_cert mirror_registry_host | ||
| client_ca_cert=/var/run/vault/mirror-registry/client_ca.crt | ||
| mirror_registry_host=$(echo "${mirror_registry}" | cut -d : -f 1) | ||
|
|
||
| # Check if registry-config ConfigMap already exists and patch or create | ||
| if oc get configmap registry-config -n openshift-config &>/dev/null; then | ||
| log "registry-config ConfigMap exists, patching with mirror registry CA..." | ||
| oc create configmap registry-config --from-file="${mirror_registry_host}..5000"=${client_ca_cert} -n openshift-config --dry-run=client -o yaml | oc apply -f - | ||
| else | ||
| log "Creating registry-config ConfigMap with mirror registry CA..." | ||
| oc create configmap registry-config --from-file="${mirror_registry_host}..5000"=${client_ca_cert} -n openshift-config | ||
| fi |
Contributor
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Find what port the bastion mirror registry in this pipeline actually uses.
rg -nP --type=sh -C3 'mirror_registry_url' ci-operator/step-registry | head -80
rg -nP --type=sh -C2 '\.\.5000|\.\.6001|\.\.6002|\.\.8443' ci-operator/step-registry | head -40Repository: openshift/release
Length of output: 17559
Hardcoded ..5000 in registry-config ConfigMap key will break non-default bastion ports.
mirror_registry is read from ${SHARED_DIR}/mirror_registry_url (format <public_dns>:<port>), then the host is split off at line 313 — but the ConfigMap key written at lines 318/321 hardcodes port 5000 ("${mirror_registry_host}..5000"). If the bastion registry listens on any other port, the cluster will not trust the actual <host>:<port> used by the IDMS/CatalogSource, causing x509/unknown-authority failures on every catalog pod and image pull from the mirror.
🛡️ Derive the port from mirror_registry instead of hardcoding
- local client_ca_cert mirror_registry_host
+ local client_ca_cert mirror_registry_host mirror_registry_port registry_config_key
client_ca_cert=/var/run/vault/mirror-registry/client_ca.crt
- mirror_registry_host=$(echo "${mirror_registry}" | cut -d : -f 1)
+ mirror_registry_host="${mirror_registry%%:*}"
+ mirror_registry_port="${mirror_registry##*:}"
+ registry_config_key="${mirror_registry_host}..${mirror_registry_port}"
# Check if registry-config ConfigMap already exists and patch or create
if oc get configmap registry-config -n openshift-config &>/dev/null; then
log "registry-config ConfigMap exists, patching with mirror registry CA..."
- oc create configmap registry-config --from-file="${mirror_registry_host}..5000"=${client_ca_cert} -n openshift-config --dry-run=client -o yaml | oc apply -f -
+ oc create configmap registry-config --from-file="${registry_config_key}=${client_ca_cert}" -n openshift-config --dry-run=client -o yaml | oc apply -f -
else
log "Creating registry-config ConfigMap with mirror registry CA..."
- oc create configmap registry-config --from-file="${mirror_registry_host}..5000"=${client_ca_cert} -n openshift-config
+ oc create configmap registry-config --from-file="${registry_config_key}=${client_ca_cert}" -n openshift-config
fi🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@ci-operator/step-registry/openshift/windows/setup-wmco-konflux-disconnected/openshift-windows-setup-wmco-konflux-disconnected-commands.sh`
around lines 309 - 322, The ConfigMap key is hardcoded to port 5000
("${mirror_registry_host}..5000") which breaks non-default ports; update the
block that sets mirror_registry_host/client_ca_cert to also extract the port
from mirror_registry (e.g., mirror_registry_port="$(echo "${mirror_registry}" |
cut -d: -f2)" or via parameter expansion), build the registry key using both
host and port ("${mirror_registry_host}:${mirror_registry_port}" or default to
5000 only if port is empty), and replace the two oc create configmap invocations
(the lines that pass --from-file with "${mirror_registry_host}..5000") to use
the derived host:port key so the ConfigMap matches the actual mirror_registry
value used by IDMS/CatalogSource.
a464aaa to
4c86fab
Compare
Contributor
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: rrasouli The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Contributor
Author
|
/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.20-amd64-nightly-vsphere-ipi-disconnected-ovn-winc-f28 periodic-ci-openshift-openshift-tests-private-release-4.21-amd64-nightly-vsphere-ipi-disconnected-ovn-winc-f28 periodic-ci-openshift-openshift-tests-private-release-4.22-amd64-nightly-vsphere-ipi-disconnected-ovn-winc-f28 |
Contributor
|
@rrasouli: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
Contributor
Author
|
… On Sun, Apr 19, 2026 at 10:58 AM coderabbitai[bot] ***@***.***> wrote:
***@***.***[bot]* commented on this pull request.
------------------------------
In
ci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.20-periodics.yaml
<#78004?email_source=notifications&email_token=AD6PAYZPCRGZOSSRNPG3D334WSBJDA5CNFSNUABKM5UWIORPF5TWS5BNNB2WEL2QOVWGYUTFOF2WK43UKJSXM2LFO4XTIMJTGU3DKNBSGYY2M4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJL3QOJPXEZLWNFSXOX3DNRUWG2Y#discussion_r3106472116>
:
> + - mountPath: /secrets/gcs
+ name: gcs-credentials
+ readOnly: true
+ - mountPath: /usr/local/github-credentials
+ name: github-credentials-openshift-ci-robot-private-git-cloner
+ readOnly: true
+ - mountPath: /secrets/manifest-tool
+ name: manifest-tool-local-pusher
+ readOnly: true
+ - mountPath: /etc/pull-secret
+ name: pull-secret
+ readOnly: true
+ - mountPath: /etc/report
+ name: result-aggregator
+ readOnly: true
+ serviceAccountName: ci-operator
+ volumes:
+ - name: boskos
+ secret:
+ items:
+ - key: credentials
+ path: credentials
+ secretName: boskos-credentials
+ - name: ci-pull-credentials
+ secret:
+ secretName: ci-pull-credentials
+ - name: github-credentials-openshift-ci-robot-private-git-cloner
+ secret:
+ secretName: github-credentials-openshift-ci-robot-private-git-cloner
+ - name: manifest-tool-local-pusher
+ secret:
+ secretName: manifest-tool-local-pusher
+ - name: pull-secret
+ secret:
+ secretName: registry-pull-credentials
+ - name: result-aggregator
+ secret:
+ secretName: result-aggregator
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
gh issue create \
--repo openshift/release \
--title "prowgen: gcs-credentials volume definition missing from vsphere02 periodic job specs" \
--body "## Summary
All \`prowgen\`-generated periodic jobs targeting the \`vsphere02\` cluster include a \`gcs-credentials\` **volumeMount** in the \`ci-operator\` container but do not include a corresponding **volume definition** in the pod spec's \`volumes\` list.
## Details
Every vsphere periodic job generated for the \`vsphere02\` cluster has:
\`\`\`yaml
volumeMounts:
- mountPath: /secrets/gcs
name: gcs-credentials
readOnly: true
\`\`\`
...but the \`volumes\` section never contains a matching \`gcs-credentials\` entry. The \`ci-operator\` arg \`--gcs-upload-secret=/secrets/gcs/service-account.json\` therefore references a mount backed by no volume definition.
## Affected jobs (examples)
- \`periodic-ci-openshift-openshift-tests-private-release-4.20-amd64-nightly-vsphere-ipi-ovn-winc-f7\`
- \`periodic-ci-openshift-openshift-tests-private-release-4.20-amd64-nightly-vsphere-ipi-disconnected-ovn-winc-f28\`
- (likely all vsphere02 periodic jobs generated by prowgen)
## Root cause hypothesis
The generator template or prowgen configuration for vsphere02 cluster jobs omits the \`gcs-credentials\` volume stanza. Jobs still run successfully, which suggests one of:
1. Namespace-level secret injection on the \`vsphere02\` cluster fills in the volume automatically.
2. \`ci-operator\` has fallback/graceful handling when the volume is unavailable.
3. The prowgen template intentionally skips it for vsphere jobs and relies on an alternative mechanism.
## Steps to reproduce
Inspect any generated periodic YAML under \`ci-operator/jobs/\` targeting \`cluster: vsphere02\` and compare \`volumeMounts\` vs \`volumes\` for the \`gcs-credentials\` entry.
## References
- PR that surfaced this pattern: #78004
- Review comment: #78004 (comment)
/cc ***@***.***`" \
--assignee rrasouli
Length of output: 54
------------------------------
@rrasouli, unfortunately the openshift/release repository has GitHub
Issues disabled, so I'm unable to create the issue there directly.
Here are the recommended alternatives for reaching the DPTP (Developer
Productivity & Testing Platform) team:
1.
*File an issue in openshift/ci-tools
<https://github.com/openshift/ci-tools/issues>* — this is the repo
that owns prowgen and the job-generation tooling. That would be the
most appropriate place to track a generator bug.
2.
*DPTP Jira project* — if your team has access, file a ticket under the
DPTP project in the Red Hat Jira instance, referencing:
- PR: #78004 <#78004>
- Comment: #78004 (comment)
<#78004 (comment)>
3.
*Slack* — ping the team in #forum-testplatform on the Red Hat CoreOS
Slack with the details above.
Here's a ready-made issue body you can paste wherever you file it:
------------------------------
*Title:* prowgen: gcs-credentials volume definition missing from
vsphere02 periodic job specs
*Body:*
All prowgen-generated periodic jobs targeting the vsphere02 cluster
include a gcs-credentials volumeMount in the ci-operator container but
have no corresponding volume definition in the pod spec's volumes list.
Example affected jobs:
-
periodic-ci-openshift-openshift-tests-private-release-4.20-amd64-nightly-vsphere-ipi-ovn-winc-f7
-
periodic-ci-openshift-openshift-tests-private-release-4.20-amd64-nightly-vsphere-ipi-disconnected-ovn-winc-f28
- (likely all vsphere02 periodic jobs generated by prowgen)
Jobs run successfully regardless, suggesting namespace-level secret
injection on vsphere02 or a prowgen intentional omission. Worth
investigating whether this is by design.
Surfaced in: #78004 <#78004> / #78004
(comment)
<#78004 (comment)>
—
Reply to this email directly, view it on GitHub
<#78004?email_source=notifications&email_token=AD6PAYZVPMG4AA3WO6THWUD4WSBJDA5CNFSNUABKM5UWIORPF5TWS5BNNB2WEL2QOVWGYUTFOF2WK43UKJSXM2LFO4XTIMJTGU3DKNBSGYY2M4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJPLQOJPXEZLWNFSXOX3ON52GSZTJMNQXI2LPNZZV6Y3MNFRWW#discussion_r3106472116>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AD6PAY3CFMMOHBKKFPUYK6D4WSBJDAVCNFSM6AAAAACX6L6H7OVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHM2DCMZVGY2TIMRWGE>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Contributor
Author
|
/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.21-amd64-nightly-vsphere-ipi-disconnected-ovn-winc-f28 |
Contributor
|
@rrasouli: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
rrasouli
force-pushed
the
vsphere-disconnected-winc-jobs
branch
from
4c86fab to
d1f3926
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In
`@ci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.22__amd64-nightly.yaml`:
- Around line 91-118: The variant entries (multi-latest.candidate.stream,
n1minor.candidate.stream, n2minor.candidate.stream, n3minor.candidate.stream,
and target.candidate.stream) erroneously use "ci" and will make the
amd64-nightly config test CI payloads; change those candidate.stream values back
to "nightly" so the nightly variant continues to target nightly payload streams
and ensure only the WINC job uses "ci" if intended.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 85dbc201-ba06-4ddb-9f4a-a9f3854b15b7
📒 Files selected for processing (10)
ci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.18.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.18__amd64-nightly.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.19.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.19__amd64-nightly.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.20.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.20__amd64-nightly.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.21.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.21__amd64-nightly.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.22.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.22__amd64-nightly.yaml
🚧 Files skipped from review as they are similar to previous changes (6)
- ci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.19.yaml
- ci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.22.yaml
- ci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.19__amd64-nightly.yaml
- ci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.18.yaml
- ci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.18__amd64-nightly.yaml
- ci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.20__amd64-nightly.yaml
Comment on lines
+91
to
+118
| stream: ci | ||
| version: "4.22" | ||
| multi-latest: | ||
| candidate: | ||
| architecture: multi | ||
| product: ocp | ||
| stream: nightly | ||
| stream: ci | ||
| version: "4.22" | ||
| n1minor: | ||
| candidate: | ||
| product: ocp | ||
| stream: nightly | ||
| stream: ci | ||
| version: "4.21" | ||
| n2minor: | ||
| candidate: | ||
| product: ocp | ||
| stream: nightly | ||
| stream: ci | ||
| version: "4.20" | ||
| n3minor: | ||
| candidate: | ||
| product: ocp | ||
| stream: nightly | ||
| stream: ci | ||
| version: "4.19" | ||
| target: | ||
| candidate: | ||
| architecture: multi | ||
| product: ocp | ||
| stream: nightly | ||
| stream: ci |
Contributor
There was a problem hiding this comment.
Keep this nightly variant on nightly payload streams.
Changing every candidate release stream to ci affects the whole amd64-nightly config, not just the new WINC job. That would make these periodic jobs test CI payloads instead of the intended nightly payloads.
🔧 Proposed fix
- stream: ci
+ stream: nightly
version: "4.22"
@@
- stream: ci
+ stream: nightly
version: "4.22"
@@
- stream: ci
+ stream: nightly
version: "4.21"
@@
- stream: ci
+ stream: nightly
version: "4.20"
@@
- stream: ci
+ stream: nightly
version: "4.19"
@@
- stream: ci
+ stream: nightly
version: "4.22"📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| stream: ci | |
| version: "4.22" | |
| multi-latest: | |
| candidate: | |
| architecture: multi | |
| product: ocp | |
| stream: nightly | |
| stream: ci | |
| version: "4.22" | |
| n1minor: | |
| candidate: | |
| product: ocp | |
| stream: nightly | |
| stream: ci | |
| version: "4.21" | |
| n2minor: | |
| candidate: | |
| product: ocp | |
| stream: nightly | |
| stream: ci | |
| version: "4.20" | |
| n3minor: | |
| candidate: | |
| product: ocp | |
| stream: nightly | |
| stream: ci | |
| version: "4.19" | |
| target: | |
| candidate: | |
| architecture: multi | |
| product: ocp | |
| stream: nightly | |
| stream: ci | |
| stream: nightly | |
| version: "4.22" | |
| multi-latest: | |
| candidate: | |
| architecture: multi | |
| product: ocp | |
| stream: nightly | |
| version: "4.22" | |
| n1minor: | |
| candidate: | |
| product: ocp | |
| stream: nightly | |
| version: "4.21" | |
| n2minor: | |
| candidate: | |
| product: ocp | |
| stream: nightly | |
| version: "4.20" | |
| n3minor: | |
| candidate: | |
| product: ocp | |
| stream: nightly | |
| version: "4.19" | |
| target: | |
| candidate: | |
| architecture: multi | |
| product: ocp | |
| stream: nightly |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@ci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.22__amd64-nightly.yaml`
around lines 91 - 118, The variant entries (multi-latest.candidate.stream,
n1minor.candidate.stream, n2minor.candidate.stream, n3minor.candidate.stream,
and target.candidate.stream) erroneously use "ci" and will make the
amd64-nightly config test CI payloads; change those candidate.stream values back
to "nightly" so the nightly variant continues to target nightly payload streams
and ensure only the WINC job uses "ci" if intended.
Contributor
Author
|
/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.21-amd64-nightly-vsphere-ipi-disconnected-ovn-winc-f28 |
Contributor
|
@rrasouli: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
Contributor
Author
|
/retest |
Contributor
Author
|
/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.20-amd64-nightly-vsphere-ipi-disconnected-ovn-winc-f28 |
Contributor
|
@rrasouli: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
- Add periodic job vsphere-ipi-disconnected-ovn-winc-f28 for releases 4.18-4.22 - Add optional presubmit debug jobs for faster PR validation - Create new workflow cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc - Mirror WMCO Konflux FBC catalog and related images to bastion registry - Translate both bundle AND operator images to Konflux sources (not registry.redhat.io) - Configure OWNERS files with Windows team aliases and symlinks - Stagger cron schedules across releases (02:00, 06:00, 10:00, 14:00, 18:00 UTC) JIRA: WINC-1611
rrasouli
force-pushed
the
vsphere-disconnected-winc-jobs
branch
from
d1f3926 to
a6bc643
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In
`@ci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.21__amd64-nightly.yaml`:
- Around line 3423-3436: The job having workflow
'cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc' currently uses
cron: "0 14 15 * *" which schedules only on the 15th of each month; update the
cron value under that job's cron key to "0 14 * * *" to run nightly at 14:00 UTC
instead.
In
`@ci-operator/step-registry/cucushift/winc/prepare/cucushift-winc-prepare-commands.sh`:
- Around line 300-315: The polling loop that waits for ready replicas (while [[
$(oc -n openshift-machine-api get machineset/${winworker_machineset_name} -o
'jsonpath={.status.readyReplicas}') != "${winworker_machineset_replicas}" ]])
must be bounded; add a timeout (e.g. record a start time and break if elapsed >
MAX_WAIT_SECONDS or use the timeout command) so the loop exits with failure if
machineset readiness isn’t reached within the bound, and on timeout print the
same diagnostics (oc get machineset, oc get machines | grep winworker, and oc
describe for each machine) and exit non-zero; reference the variables
winworker_machineset_name and winworker_machineset_replicas so the change wraps
that loop and triggers the existing diagnostic block on timeout.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 349a5abe-d94e-4f2e-b16b-4131caa6b404
📒 Files selected for processing (42)
ci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.18.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.18__amd64-nightly.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.19.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.19__amd64-nightly.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.20.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.20__amd64-nightly.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.21.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.21__amd64-nightly.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.22.yamlci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.22__amd64-nightly.yamlci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.18-periodics.yamlci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.18-presubmits.yamlci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.19-periodics.yamlci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.19-presubmits.yamlci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.20-periodics.yamlci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.20-presubmits.yamlci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.21-periodics.yamlci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.21-presubmits.yamlci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.22-periodics.yamlci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.22-presubmits.yamlci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/OWNERSci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/OWNERSci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-workflow.metadata.jsonci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-workflow.yamlci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/deprovision/OWNERSci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/deprovision/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-deprovision-chain.metadata.jsonci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/deprovision/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-deprovision-chain.yamlci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/provision/OWNERSci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/provision/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-provision-chain.metadata.jsonci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/provision/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-provision-chain.yamlci-operator/step-registry/cucushift/winc/prepare/cucushift-winc-prepare-commands.shci-operator/step-registry/openshift/windows/OWNERSci-operator/step-registry/openshift/windows/install-wmco/OWNERSci-operator/step-registry/openshift/windows/install-wmco/openshift-windows-install-wmco-ref.metadata.jsonci-operator/step-registry/openshift/windows/setup-wmco-konflux-disconnected/OWNERSci-operator/step-registry/openshift/windows/setup-wmco-konflux-disconnected/openshift-windows-setup-wmco-konflux-disconnected-commands.shci-operator/step-registry/openshift/windows/setup-wmco-konflux-disconnected/openshift-windows-setup-wmco-konflux-disconnected-ref.metadata.jsonci-operator/step-registry/openshift/windows/setup-wmco-konflux-disconnected/openshift-windows-setup-wmco-konflux-disconnected-ref.yamlci-operator/step-registry/openshift/windows/setup-wmco-konflux/OWNERSci-operator/step-registry/openshift/windows/setup-wmco-konflux/openshift-windows-setup-wmco-konflux-ref.metadata.jsonci-operator/step-registry/openshift/windows/winc-workflow/OWNERSci-operator/step-registry/openshift/windows/winc-workflow/openshift-windows-winc-workflow-chain.metadata.json
💤 Files with no reviewable changes (7)
- ci-operator/step-registry/openshift/windows/OWNERS
- ci-operator/step-registry/openshift/windows/winc-workflow/openshift-windows-winc-workflow-chain.metadata.json
- ci-operator/step-registry/openshift/windows/install-wmco/OWNERS
- ci-operator/step-registry/openshift/windows/setup-wmco-konflux/OWNERS
- ci-operator/step-registry/openshift/windows/winc-workflow/OWNERS
- ci-operator/step-registry/openshift/windows/setup-wmco-konflux/openshift-windows-setup-wmco-konflux-ref.metadata.json
- ci-operator/step-registry/openshift/windows/install-wmco/openshift-windows-install-wmco-ref.metadata.json
✅ Files skipped from review due to trivial changes (13)
- ci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/OWNERS
- ci-operator/step-registry/openshift/windows/setup-wmco-konflux-disconnected/OWNERS
- ci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/provision/OWNERS
- ci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/OWNERS
- ci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/deprovision/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-deprovision-chain.yaml
- ci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-workflow.metadata.json
- ci-operator/step-registry/openshift/windows/setup-wmco-konflux-disconnected/openshift-windows-setup-wmco-konflux-disconnected-ref.metadata.json
- ci-operator/step-registry/openshift/windows/setup-wmco-konflux-disconnected/openshift-windows-setup-wmco-konflux-disconnected-ref.yaml
- ci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-workflow.yaml
- ci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/deprovision/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-deprovision-chain.metadata.json
- ci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/deprovision/OWNERS
- ci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/provision/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-provision-chain.metadata.json
- ci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.21-periodics.yaml
🚧 Files skipped from review as they are similar to previous changes (13)
- ci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.19.yaml
- ci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.20-presubmits.yaml
- ci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.22-presubmits.yaml
- ci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.18-periodics.yaml
- ci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.20-periodics.yaml
- ci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.19-periodics.yaml
- ci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.22-periodics.yaml
- ci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.20__amd64-nightly.yaml
- ci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.22__amd64-nightly.yaml
- ci-operator/step-registry/openshift/windows/setup-wmco-konflux-disconnected/openshift-windows-setup-wmco-konflux-disconnected-commands.sh
- ci-operator/jobs/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.18-presubmits.yaml
- ci-operator/step-registry/cucushift/installer/rehearse/vsphere/ipi/disconnected/ovn/winc/provision/cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc-provision-chain.yaml
- ci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.18__amd64-nightly.yaml
Comment on lines
+3423
to
+3436
| - as: vsphere-ipi-disconnected-ovn-winc-f28 | ||
| cron: 0 14 15 * * | ||
| steps: | ||
| cluster_profile: vsphere-dis-2 | ||
| env: | ||
| TEST_FILTERS: ~ChkUpgrade&;~ConnectedOnly&;Smokerun& | ||
| TEST_SCENARIOS: Windows_Containers | ||
| TEST_TIMEOUT: "50" | ||
| leases: | ||
| - env: VSPHERE_BASTION_LEASED_RESOURCE | ||
| resource_type: vsphere-connected-2-quota-slice | ||
| test: | ||
| - ref: openshift-extended-test | ||
| workflow: cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc |
Contributor
There was a problem hiding this comment.
Use a daily cron if this is meant to be a nightly periodic.
0 14 15 * * runs only on the 15th day of each month. If the target cadence is the 14:00 UTC nightly slot, this should be 0 14 * * *.
Suggested cron change for nightly cadence
- as: vsphere-ipi-disconnected-ovn-winc-f28
- cron: 0 14 15 * *
+ cron: 0 14 * * *🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@ci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.21__amd64-nightly.yaml`
around lines 3423 - 3436, The job having workflow
'cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-winc' currently uses
cron: "0 14 15 * *" which schedules only on the 15th of each month; update the
cron value under that job's cron key to "0 14 * * *" to run nightly at 14:00 UTC
instead.
Comment on lines
+300
to
+315
| echo "DEBUG: Windows machineset configuration:" | ||
| oc get machineset -n openshift-machine-api ${winworker_machineset_name} -o yaml | ||
|
|
||
| echo "Waiting for ${winworker_machineset_replicas} Windows machines to become ready..." | ||
| while [[ $(oc -n openshift-machine-api get machineset/${winworker_machineset_name} -o 'jsonpath={.status.readyReplicas}') != "${winworker_machineset_replicas}" ]]; do echo -n "." && sleep 10; done | ||
| echo "" | ||
| echo "${winworker_machineset_replicas} Windows machines are ready in machineset" | ||
|
|
||
| echo "DEBUG: Windows machines status:" | ||
| oc get machines -n openshift-machine-api | grep winworker || echo "No Windows machines found" | ||
|
|
||
| echo "DEBUG: Describing Windows machines:" | ||
| for machine in $(oc get machines -n openshift-machine-api -o name | grep winworker); do | ||
| echo "=== ${machine} ===" | ||
| oc describe -n openshift-machine-api ${machine} | ||
| done |
Contributor
There was a problem hiding this comment.
Bound the machineset readiness wait before the step times out.
Line 304 can loop until the 40-minute step timeout kills the container, skipping the new node/workload diagnostics. Add an explicit timeout around this poll so failures emit the machineset and machine details you already collect.
Suggested bounded wait
echo "Waiting for ${winworker_machineset_replicas} Windows machines to become ready..."
-while [[ $(oc -n openshift-machine-api get machineset/${winworker_machineset_name} -o 'jsonpath={.status.readyReplicas}') != "${winworker_machineset_replicas}" ]]; do echo -n "." && sleep 10; done
+deadline=$((SECONDS + 1800))
+while [[ $(oc -n openshift-machine-api get "machineset/${winworker_machineset_name}" -o 'jsonpath={.status.readyReplicas}') != "${winworker_machineset_replicas}" ]]; do
+ if (( SECONDS >= deadline )); then
+ echo ""
+ echo "ERROR: Timed out waiting for ${winworker_machineset_replicas} Windows machines to become ready"
+ oc get machineset -n openshift-machine-api "${winworker_machineset_name}" -o yaml
+ oc get machines -n openshift-machine-api -o wide | grep winworker || true
+ exit 1
+ fi
+ echo -n "."
+ sleep 10
+done🧰 Tools
🪛 Shellcheck (0.11.0)
[info] 301-301: Double quote to prevent globbing and word splitting.
(SC2086)
[info] 304-304: Double quote to prevent globbing and word splitting.
(SC2086)
[info] 314-314: Double quote to prevent globbing and word splitting.
(SC2086)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
`@ci-operator/step-registry/cucushift/winc/prepare/cucushift-winc-prepare-commands.sh`
around lines 300 - 315, The polling loop that waits for ready replicas (while [[
$(oc -n openshift-machine-api get machineset/${winworker_machineset_name} -o
'jsonpath={.status.readyReplicas}') != "${winworker_machineset_replicas}" ]])
must be bounded; add a timeout (e.g. record a start time and break if elapsed >
MAX_WAIT_SECONDS or use the timeout command) so the loop exits with failure if
machineset readiness isn’t reached within the bound, and on timeout print the
same diagnostics (oc get machineset, oc get machines | grep winworker, and oc
describe for each machine) and exit non-zero; reference the variables
winworker_machineset_name and winworker_machineset_replicas so the change wraps
that loop and triggers the existing diagnostic block on timeout.
Contributor
Author
|
/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.21-amd64-nightly-vsphere-ipi-disconnected-ovn-winc-f28 |
Contributor
|
@rrasouli: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
Contributor
Author
|
/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.20-amd64-nightly-vsphere-ipi-disconnected-ovn-winc-f28 |
Contributor
|
@rrasouli: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
Contributor
Author
|
/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.20-amd64-nightly-vsphere-ipi-disconnected-ovn-winc-f28 |
Contributor
|
@rrasouli: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
Contributor
Author
|
/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.20-amd64-nightly-vsphere-ipi-disconnected-ovn-winc-f28 |
Contributor
|
@rrasouli: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
Contributor
Author
|
/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.19-amd64-nightly-vsphere-ipi-disconnected-ovn-winc-f28 |
Contributor
|
@rrasouli: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
Contributor
Author
|
/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.19-amd64-nightly-vsphere-ipi-disconnected-ovn-winc-f28 |
Contributor
|
@rrasouli: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
Contributor
|
@rrasouli: requesting more than one rehearsal in one comment is not supported. If you would like to rehearse multiple specific jobs, please separate the job names by a space in a single command. |
Contributor
|
@rrasouli: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
Contributor
Author
|
Closing in favor of a working #76900 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add vSphere IPI disconnected Windows Containers testing jobs for OCP releases 4.18-4.22:
Changes
New workflow:
cucushift-installer-rehearse-vsphere-ipi-disconnected-ovn-wincKey features:
Test coverage:
Test plan
Notes
JIRA: WINC-1611
Summary by CodeRabbit
New Features
Bug Fixes
Chores