Skip to content

[WIP] refactor(ci): migrate HyperShift calico private workflow from cucushift to hypershift step registry#80907

Open
mgencur wants to merge 3 commits into
openshift:mainfrom
mgencur:calico_4.22_migrate_steps
Open

[WIP] refactor(ci): migrate HyperShift calico private workflow from cucushift to hypershift step registry#80907
mgencur wants to merge 3 commits into
openshift:mainfrom
mgencur:calico_4.22_migrate_steps

Conversation

@mgencur

@mgencur mgencur commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Related slack discussion: https://redhat-internal.slack.com/archives/G01QS0P2F6W/p1782211558060659?thread_ts=1782207388.083069&cid=G01QS0P2F6W

Summary by CodeRabbit

This PR refactors the HyperShift Calico conformance testing infrastructure in OpenShift CI by migrating from cucushift-based step registry components to a new hypershift-native component hierarchy. The migration improves code organization and consolidates duplicate configuration across multiple OpenShift versions (4.19-4.22).

Core Changes

Workflow Migration for Calico Conformance Testing:

  • The periodic job e2e-aws-conformance-calico-private across OpenShift versions 4.19-4.22 is updated to use the new hypershift-aws-conformance-calico-private workflow instead of the previous cucushift-installer-rehearse-aws-ipi-ovn-hypershift-private-guest-calico workflow
  • Configuration parameters (TEST_ARGS, TEST_SKIPS) that were previously specified inline in individual CI job configs are now consolidated into the workflow definitions themselves, eliminating redundant specifications
  • OpenShift 4.22 receives three new periodic conformance test jobs: e2e-aws-conformance-calico, e2e-aws-conformance-calico-private, and e2e-kubevirt-metal-conformance-calico

New HyperShift Step Registry Components:
The PR introduces a complete set of new step registry components organized under the hypershift/ directory:

  1. Calico CNI Support (hypershift/calico/):

    • install: Downloads and applies the Tigera Calico operator, configures AWS credentials when needed, and performs the full Calico installation
    • health-check: Validates Calico deployment health by checking tigerastatus components, CRD availability, cluster operators readiness, and node readiness
  2. AWS Private Cluster Infrastructure (hypershift/aws/):

    • install-private: Creates an S3 bucket for OIDC documents and installs the HyperShift operator with platform-specific configurations
    • metadata: Retrieves VPC ID and public subnet information from the hosted cluster and persists them for downstream steps
    • private-provision: Orchestrates the complete provisioning chain for private hosted clusters including installer setup, operator installation, cluster creation, metadata discovery, bastion provisioning, and proxy configuration
    • private-deprovision: Handles cleanup and deprovisioning of private cluster resources with proper dependency ordering
  3. Guest Cluster Management (hypershift/enable-guest/, hypershift/disable-guest/):

    • enable-guest: Switches from management cluster kubeconfig to guest cluster kubeconfig and extracts the OpenShift console URL
    • disable-guest: Reverts to management cluster kubeconfig for post-test operations
  4. QE Enablement (hypershift/enable-qe/):

    • pull-secret: Performs day-2 pull-secret updates with verification that MachineDeployment rollouts complete successfully before testing
    • catalogsource: Installs QE catalog sources in the OpenShift Marketplace for operator testing

Technical Improvements

  • CNI Provider Support Enhancement: The AWS create-hostedcluster step now supports both cilium and calico as CNI providers (previously cilium-only), enabling the load balancer health probe annotation for both
  • Pull Secret Update Flow: Added comprehensive rollout monitoring to ensure all machine deployments complete their updates before proceeding with conformance tests
  • Modular Architecture: Breaking the monolithic cucushift workflow into discrete, reusable hypershift components improves testability and allows better reuse across different cluster configurations
  • Step Addition in 4.22: The existing cucushift Calico workflow receives an environment variable CNI_PROVIDER: "calico" for consistency

Impact

This refactoring centralizes test configuration management, reduces CI configuration duplication across OpenShift versions, and establishes a cleaner component-based architecture for HyperShift Calico conformance testing that can be more easily maintained and extended in the future.

mgencur and others added 2 commits June 23, 2026 13:25
Port three calico conformance tests from 4.21 to the 4.22 periodics
config: e2e-aws-conformance-calico, e2e-aws-conformance-calico-private,
and e2e-kubevirt-metal-conformance-calico. LVM operator bumped to
stable-4.22; ODF kept at stable-4.21.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ft to hypershift step registry

Create new step registry components under hypershift/ for the private
Calico conformance workflow, replacing cucushift-prefixed equivalents.
Move TEST_SKIPS and TEST_ARGS from CI configs into the workflow itself.
Update all calico ref references across hypershift workflows.

Components migrated:
- hypershift-calico-install, hypershift-calico-health-check (refs)
- hypershift-aws-install-private (ref + chain)
- hypershift-aws-metadata, hypershift-enable/disable-guest (refs)
- hypershift-enable-qe-catalogsource (ref + chain)
- hypershift-aws-private-provision/deprovision (chains)
- hypershift-aws-conformance-calico-private (workflow)

CI configs updated: 4.19-4.22 periodics now use the new workflow.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 23, 2026
@coderabbitai

coderabbitai Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Walkthrough

The PR introduces a new hypershift-aws-conformance-calico-private CI workflow for running OpenShift conformance tests against private HyperShift AWS clusters using Calico CNI. It adds native step-registry primitives for Calico installation, health-checking, AWS private cluster provisioning/deprovisioning, guest kubeconfig switching, and QE enablement, then wires these into the 4.19–4.22 periodic job configs while migrating existing calico workflows from legacy cucushift steps.

Changes

HyperShift Calico Private Conformance

Layer / File(s) Summary
Calico install and health-check primitives
ci-operator/step-registry/hypershift/calico/install/..., ci-operator/step-registry/hypershift/calico/health-check/...
New hypershift-calico-install step downloads the Tigera operator tarball, creates AWS credentials secret, bootstraps CRDs, and applies the Calico Installation CR with NodeInternalIP autodetection. New hypershift-calico-health-check step polls node count and waits on tigerastatus, clusteroperator, and clusterversion conditions.
AWS private cluster infrastructure steps and chains
ci-operator/step-registry/hypershift/aws/install-private/..., ci-operator/step-registry/hypershift/aws/metadata/..., ci-operator/step-registry/hypershift/aws/private-provision/..., ci-operator/step-registry/hypershift/aws/private-deprovision/...
New hypershift-aws-install-private-config creates a public S3 bucket for OIDC. New hypershift-aws-install-private composes the hypershift install command with OIDC/private/external-DNS flags. New hypershift-aws-metadata fetches VPC and public subnet IDs. Provision and deprovision chains compose all sub-steps with documented ordering (bastion before hosted cluster).
Guest kubeconfig switching steps
ci-operator/step-registry/hypershift/enable-guest/..., ci-operator/step-registry/hypershift/disable-guest/...
New hypershift-enable-guest saves management kubeconfig, switches to nested guest kubeconfig, writes the console URL and cluster-type marker. New hypershift-disable-guest restores management kubeconfig.
QE pull-secret update and catalog source steps
ci-operator/step-registry/hypershift/enable-qe/pull-secret/..., ci-operator/step-registry/hypershift/enable-qe/catalogsource/..., ci-operator/step-registry/cucushift/hypershift-extended/enable-qe/pull-secret/cucushift-hypershift-extended-enable-qe-pull-secret-commands.sh
New hypershift-enable-qe-pull-secret patches the HostedCluster pull secret from Vault credentials, waits for MachineDeployment rollouts, and verifies each worker node via day-2 loop. New hypershift-enable-qe-catalogsource ensures openshift-marketplace namespace and creates qe-app-registry CatalogSource. The existing cucushift pull-secret script receives the same MachineDeployment rollout wait.
Conformance-calico workflows and CNI_PROVIDER support
ci-operator/step-registry/hypershift/aws/conformance-calico-private/..., ci-operator/step-registry/hypershift/aws/conformance-calico/hypershift-aws-conformance-calico-workflow.yaml, ci-operator/step-registry/hypershift/aws/create/hypershift-aws-create-chain.yaml, ci-operator/step-registry/hypershift/kubevirt/baremetalds/conformance-calico/..., ci-operator/step-registry/hypershift/mce/agent/metal3/create/calico/..., ci-operator/step-registry/cucushift/installer/rehearse/aws/.../calico/...
New hypershift-aws-conformance-calico-private workflow defines the full private cluster lifecycle. Updated hypershift-aws-conformance-calico adds CNI_PROVIDER: calico, revised TEST_SKIPS, and switches pre-step refs. hypershift-aws-create-chain extends load-balancer annotation conditional to cover calico. Kubevirt and MCE metal3 calico chains migrate from cucushift to new hypershift step refs.
Periodic job wiring for 4.19–4.22
ci-operator/config/openshift/hypershift/openshift-hypershift-release-4.19__periodics.yaml, ...4.20..., ...4.21..., ...4.22...
The e2e-aws-conformance-calico-private job in 4.19/4.20/4.21 is updated to reference the new workflow, dropping inline test customization. Three new periodic jobs are added to 4.22: e2e-aws-conformance-calico, e2e-aws-conformance-calico-private, and e2e-kubevirt-metal-conformance-calico.

Sequence Diagram(s)

sequenceDiagram
  participant Prow as Prow Scheduler
  participant Provision as hypershift-aws-private-provision chain
  participant S3Config as hypershift-aws-install-private-config
  participant HCPInstall as hypershift-aws-install-private
  participant CalicoInstall as hypershift-calico-install
  participant CalicoHC as hypershift-calico-health-check
  participant EnableGuest as hypershift-enable-guest
  participant QEPullSecret as hypershift-enable-qe-pull-secret
  participant Conformance as openshift-conformance-suite
  participant Deprovision as hypershift-aws-private-deprovision chain

  Prow->>Provision: pre-steps: provision private cluster
  Provision->>S3Config: create S3 OIDC bucket
  Provision->>HCPInstall: hypershift install (private, OIDC S3, external-DNS)
  Provision->>CalicoInstall: apply Tigera operator + CRDs + Installation CR
  CalicoInstall-->>Provision: Calico installed
  Provision->>CalicoHC: wait nodes/tigerastatus/clusteroperators ready
  CalicoHC-->>Provision: cluster healthy
  Provision->>EnableGuest: swap kubeconfig to nested_kubeconfig, write console URL
  Provision->>QEPullSecret: patch HostedCluster pull secret, wait MachineDeployment rollout, verify nodes
  Prow->>Conformance: test: run conformance with TEST_SKIPS
  Prow->>Deprovision: post: dump state, destroy AWS resources, ipi-deprovision
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Suggested labels

lgtm, approved, rehearsals-ack

Suggested reviewers

  • csrwng

Important

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

❌ Failed checks (2 errors, 1 warning)

Check name Status Explanation Resolution
Stable And Deterministic Test Names ❌ Error Test skip patterns in hypershift-aws-conformance-calico-private/private-workflow.yaml include "net.ipv4.conf.IFNAME.arp_filter" where IFNAME is a runtime token replaced with actual interface names,... Replace IFNAME placeholder in test names with a static, descriptive string that doesn't change between runs, or use a wildcard regex pattern in TEST_SKIPS instead of embedding the placeholder in the test title.
No-Sensitive-Data-In-Logs ❌ Error Three new scripts expose sensitive data in logs: (1) hypershift-enable-qe-pull-secret-commands.sh uses vault credentials in echo/base64 without set +x protection, embedding them in jq arguments (li... Disable bash tracing (set +x) before handling credentials, and re-enable after. Avoid embedding credentials/IDs in command arguments, pass via environment variables instead or use credential files directly without logging paths.
Docstring Coverage ⚠️ Warning Docstring coverage is 37.50% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (12 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically describes the main change: migrating HyperShift Calico private workflow from cucushift to hypershift step registry.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Test Structure And Quality ✅ Passed The custom check for Ginkgo test code quality is not applicable to this PR. The PR contains only YAML configs, Bash scripts, and JSON metadata for CI/operator infrastructure—no Go source files or G...
Microshift Test Compatibility ✅ Passed This PR contains no new Ginkgo e2e test files. Changes are limited to CI infrastructure: YAML job configs, shell scripts, and metadata files.
Single Node Openshift (Sno) Test Compatibility ✅ Passed This PR contains only CI infrastructure changes (YAML workflow configs and shell scripts), not Ginkgo e2e test code. No new Go test files with It(), Describe(), Context(), or When() test definition...
Topology-Aware Scheduling Compatibility ✅ Passed This PR only modifies CI/testing infrastructure (periodic job configs, workflow definitions, test scripts, and metadata), not production deployment manifests, operator code, or controllers. The top...
Ote Binary Stdout Contract ✅ Passed This PR contains only CI/operator configuration files (YAML), shell scripts, and metadata files. It contains no Go source code or test binaries, making the OTE Binary Stdout Contract check not appl...
Ipv6 And Disconnected Network Test Compatibility ✅ Passed This PR contains only CI configuration, workflows, and helper scripts—no Ginkgo e2e tests. The check only applies to new test code, making it not applicable here.
No-Weak-Crypto ✅ Passed PR adds hypershift calico workflow migration with no weak cryptography: no MD5/SHA1/DES/RC4/Blowfish/ECB detected; SHA256 is acceptable; comparisons involve only boolean flags and counts, not secrets.
Container-Privileges ✅ Passed No container privilege escalation indicators found: no privileged:true, hostPID/Network/IPC, SYS_ADMIN, allowPrivilegeEscalation:true, or runAsUser:0 without justification.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

[REHEARSALNOTIFIER]
@mgencur: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name Repo Type Reason
pull-ci-openshift-cluster-network-operator-master-e2e-aws-ovn-hypershift-conformance openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-5.1-e2e-aws-ovn-hypershift-conformance openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-5.0-e2e-aws-ovn-hypershift-conformance openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-4.23-e2e-aws-ovn-hypershift-conformance openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-4.22-e2e-aws-ovn-hypershift-conformance openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-4.21-e2e-aws-ovn-hypershift-conformance openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-4.20-e2e-aws-ovn-hypershift-conformance openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-4.19-e2e-aws-ovn-hypershift-conformance openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-4.18-e2e-aws-ovn-hypershift-conformance openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-4.17-e2e-aws-ovn-hypershift-conformance openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-4.16-e2e-aws-ovn-hypershift-conformance openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-4.15-e2e-hypershift-ovn openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-4.14-e2e-hypershift-ovn openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-4.13-e2e-hypershift-ovn openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-4.12-e2e-hypershift-ovn openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-4.11-e2e-hypershift openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-console-operator-main-e2e-hypershift-conformance openshift/console-operator presubmit Registry content changed
pull-ci-openshift-console-operator-release-5.1-e2e-hypershift-conformance openshift/console-operator presubmit Registry content changed
pull-ci-openshift-console-operator-release-5.0-e2e-hypershift-conformance openshift/console-operator presubmit Registry content changed
pull-ci-openshift-console-operator-release-4.23-e2e-hypershift-conformance openshift/console-operator presubmit Registry content changed
pull-ci-openshift-console-operator-release-4.22-e2e-hypershift-conformance openshift/console-operator presubmit Registry content changed
pull-ci-openshift-console-operator-release-4.21-e2e-hypershift-conformance openshift/console-operator presubmit Registry content changed
pull-ci-openshift-console-operator-release-4.20-e2e-hypershift-conformance openshift/console-operator presubmit Registry content changed
pull-ci-openshift-console-operator-release-4.19-e2e-hypershift-conformance openshift/console-operator presubmit Registry content changed
pull-ci-openshift-console-operator-release-4.18-e2e-hypershift-conformance openshift/console-operator presubmit Registry content changed

A total of 716 jobs have been affected by this change. The above listing is non-exhaustive and limited to 25 jobs.

A full list of affected jobs can be found here
Prior to this PR being merged, you will need to either run and acknowledge or opt to skip these rehearsals.

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 9

🧹 Nitpick comments (3)
ci-operator/step-registry/hypershift/disable-guest/hypershift-disable-guest-commands.sh (1)

3-3: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Remove default xtrace from this step script.

Line 3 enables -x by default; repository guidance for step-registry scripts prefers set -euo pipefail and enabling tracing only for active debugging.

Suggested fix
-set -xeuo pipefail
+set -euo pipefail

As per coding guidelines, "Use default set -euo pipefail (without -x) in step registry command scripts; only enable -x when actively debugging."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/hypershift/disable-guest/hypershift-disable-guest-commands.sh`
at line 3, The hypershift-disable-guest-commands.sh script contains `set -xeuo
pipefail` which enables xtrace (the `-x` flag) by default. According to
repository guidelines, remove the `-x` flag from the set command to use only
`set -euo pipefail`, reserving xtrace only for active debugging sessions rather
than enabling it by default in step registry scripts.

Source: Coding guidelines

ci-operator/step-registry/hypershift/enable-qe/catalogsource/hypershift-enable-qe-catalogsource-commands.sh (1)

3-7: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Remove unconditional xtrace from step-registry command scripts.

set -x is enabled by default. Repository guidance for ci-operator/step-registry/**/*-commands.sh requires defaulting to set -euo pipefail without -x, enabling tracing only when actively debugging.

Suggested fix
 set -e
 set -u
 set -o pipefail
-set -x
As per coding guidelines, "Use default `set -euo pipefail` (without `-x`) in step registry command scripts; only enable `-x` when actively debugging".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/hypershift/enable-qe/catalogsource/hypershift-enable-qe-catalogsource-commands.sh`
around lines 3 - 7, The hypershift-enable-qe-catalogsource-commands.sh script
unconditionally enables xtrace with set -x, which violates the step-registry
command script guidelines. Remove the set -x line from the script, keeping only
set -euo pipefail (which encompasses set -e, set -u, and set -o pipefail). The
xtrace debugging should only be enabled when actively debugging, not by default
in production scripts.

Source: Coding guidelines

ci-operator/step-registry/hypershift/aws/metadata/hypershift-aws-metadata-commands.sh (1)

5-5: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

Avoid enabling xtrace by default in step-registry scripts.

Please keep default flags to set -euo pipefail and enable tracing only when explicitly debugging. As per coding guidelines, “Use default set -euo pipefail (without -x) in step registry command scripts; only enable -x when actively debugging.”

Suggested patch
-set -x
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/hypershift/aws/metadata/hypershift-aws-metadata-commands.sh`
at line 5, Remove the `-x` flag from the `set -x` command in the
hypershift-aws-metadata-commands.sh script. Replace it with the standard default
flags `set -euo pipefail` to disable xtrace by default, as per the coding
guidelines for step-registry scripts. The `-x` flag should only be enabled when
actively debugging specific issues, not in production step-registry command
scripts.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@ci-operator/step-registry/hypershift/aws/install-private/config/hypershift-aws-install-private-config-commands.sh`:
- Around line 3-5: The shell script options need to be updated to follow
fail-fast standards and avoid unconditional debug tracing. In the set commands
at lines 3-5, replace the individual `set -o nounset` and `set -o pipefail`
statements with the combined `set -euo pipefail` which includes error exit
handling (errexit) and prevents masking of AWS command failures. Remove the
unconditional `set -x` (xtrace) from line 5 since debug tracing should be opt-in
for debugging only, not enabled by default. Additionally, on line 15 where
PROW_JOB_ID is referenced, ensure it is properly quoted to prevent
word-splitting and globbing issues.

In
`@ci-operator/step-registry/hypershift/aws/install-private/hypershift-aws-install-private-commands.sh`:
- Line 3: The shell script currently only sets `-u` option on line 3, which
fails to enable fail-fast behavior for command failures that occur before line
111. Replace the `set -u` statement with `set -euo pipefail` to properly enable
error handling, undefined variable detection, and pipe failure detection
throughout the script. This aligns with the step-registry coding guidelines that
require fail-fast shell options (without trace mode) at the beginning of command
scripts to catch failures from early CLI operations like `oc` calls.
- Line 112: The eval command used to execute the COMMAND array on line 112
causes shell re-interpretation of arguments, which means spaces and
metacharacters in environment variable values can unintentionally change the
command's behavior. Replace the eval invocation with direct array expansion
using COMMAND[@] syntax, which properly passes array elements as separate
arguments without additional shell parsing. This ensures that values containing
spaces or special characters are treated as literal strings rather than being
re-interpreted by the shell.

In
`@ci-operator/step-registry/hypershift/aws/install-private/hypershift-aws-install-private-ref.yaml`:
- Line 38: The URL in the comment on line 38 contains a malformed host format
with a colon instead of a dot. Replace the colon in
`https://issues:redhat.com/browse/NE-1298` with a dot to create the valid format
`https://issues.redhat.com/browse/NE-1298`.

In
`@ci-operator/step-registry/hypershift/aws/metadata/hypershift-aws-metadata-commands.sh`:
- Around line 11-14: Add validation checks immediately after the vpc_id,
infra_id, and public_subnet variable assignments to ensure these metadata
lookups return valid non-empty values rather than empty strings or "None". For
each variable, check if it is empty or equals "None" and if so, exit the script
with a clear error message indicating which lookup failed. This prevents invalid
values from being written to shared files that are consumed downstream by
aws-provision-bastionhost-commands.sh, ensuring failures are caught early with
clear diagnostics rather than causing obscure errors later in the provisioning
process.

In
`@ci-operator/step-registry/hypershift/calico/health-check/hypershift-calico-health-check-commands.sh`:
- Line 14: The timeout command with the nested bash shell at line 14 does not
validate that HYPERSHIFT_NODE_COUNT is set and numeric before executing, which
causes the script to wait the full 30 minute timeout if the variable is missing
or invalid. Add validation checks before the timeout command to ensure
HYPERSHIFT_NODE_COUNT is defined and contains only numeric characters, using
parameter expansion checks or explicit arithmetic validation. If validation
fails, exit immediately with a descriptive error message rather than allowing
the script to enter the timeout loop.

In
`@ci-operator/step-registry/hypershift/calico/install/hypershift-calico-install-commands.sh`:
- Line 3: Remove the `-x` flag from the `set -xeuo pipefail` command at the
beginning of the hypershift-calico-install-commands.sh script. According to repo
guidance for step-registry command scripts, the default should be `set -euo
pipefail` without xtrace enabled, and tracing should only be enabled when
actively debugging. Change the set command to remove the `x` option while
keeping the `e`, `u`, `o pipefail` options intact.

In
`@ci-operator/step-registry/hypershift/enable-guest/hypershift-enable-guest-commands.sh`:
- Line 13: The echo command on line 13 masks failures from the oc command
substitution, potentially writing a malformed URL to hostedcluster_console.url
file. Refactor this to first execute the oc command separately and store the
result in a variable, then validate that the host variable is non-empty to catch
any oc failures, and finally construct the URL string using the validated host
before writing it to the file. This prevents malformed URLs from being silently
written when the oc command fails.

In
`@ci-operator/step-registry/hypershift/enable-qe/catalogsource/hypershift-enable-qe-catalogsource-commands.sh`:
- Around line 106-109: Remove the conditional check starting at line 106 that
uses SKIP_HYPERSHIFT_PULL_SECRET_UPDATE to gate the entire catalogsource
creation step. Since this flag is documented as pull-secret-specific and not
intended to control catalogsource setup, the catalogsource creation logic should
execute independently regardless of the pull-secret skip flag status. Delete the
if statement checking SKIP_HYPERSHIFT_PULL_SECRET_UPDATE and the exit 0 call so
that the catalogsource setup proceeds unconditionally.

---

Nitpick comments:
In
`@ci-operator/step-registry/hypershift/aws/metadata/hypershift-aws-metadata-commands.sh`:
- Line 5: Remove the `-x` flag from the `set -x` command in the
hypershift-aws-metadata-commands.sh script. Replace it with the standard default
flags `set -euo pipefail` to disable xtrace by default, as per the coding
guidelines for step-registry scripts. The `-x` flag should only be enabled when
actively debugging specific issues, not in production step-registry command
scripts.

In
`@ci-operator/step-registry/hypershift/disable-guest/hypershift-disable-guest-commands.sh`:
- Line 3: The hypershift-disable-guest-commands.sh script contains `set -xeuo
pipefail` which enables xtrace (the `-x` flag) by default. According to
repository guidelines, remove the `-x` flag from the set command to use only
`set -euo pipefail`, reserving xtrace only for active debugging sessions rather
than enabling it by default in step registry scripts.

In
`@ci-operator/step-registry/hypershift/enable-qe/catalogsource/hypershift-enable-qe-catalogsource-commands.sh`:
- Around line 3-7: The hypershift-enable-qe-catalogsource-commands.sh script
unconditionally enables xtrace with set -x, which violates the step-registry
command script guidelines. Remove the set -x line from the script, keeping only
set -euo pipefail (which encompasses set -e, set -u, and set -o pipefail). The
xtrace debugging should only be enabled when actively debugging, not by default
in production scripts.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 5f4dcf11-a705-4543-9691-c19537af62cc

📥 Commits

Reviewing files that changed from the base of the PR and between 93fe74e and e3d6d68.

⛔ Files ignored due to path filters (1)
  • ci-operator/jobs/openshift/hypershift/openshift-hypershift-release-4.22-periodics.yaml is excluded by !ci-operator/jobs/**
📒 Files selected for processing (61)
  • ci-operator/config/openshift/hypershift/openshift-hypershift-release-4.19__periodics.yaml
  • ci-operator/config/openshift/hypershift/openshift-hypershift-release-4.20__periodics.yaml
  • ci-operator/config/openshift/hypershift/openshift-hypershift-release-4.21__periodics.yaml
  • ci-operator/config/openshift/hypershift/openshift-hypershift-release-4.22__periodics.yaml
  • ci-operator/step-registry/cucushift/hypershift-extended/enable-qe/pull-secret/cucushift-hypershift-extended-enable-qe-pull-secret-commands.sh
  • ci-operator/step-registry/cucushift/installer/rehearse/aws/ipi/ovn/hypershift/private/guest/calico/cucushift-installer-rehearse-aws-ipi-ovn-hypershift-private-guest-calico-workflow.yaml
  • ci-operator/step-registry/hypershift/aws/conformance-calico-private/OWNERS
  • ci-operator/step-registry/hypershift/aws/conformance-calico-private/hypershift-aws-conformance-calico-private-workflow.metadata.json
  • ci-operator/step-registry/hypershift/aws/conformance-calico-private/hypershift-aws-conformance-calico-private-workflow.yaml
  • ci-operator/step-registry/hypershift/aws/conformance-calico/hypershift-aws-conformance-calico-workflow.yaml
  • ci-operator/step-registry/hypershift/aws/create/hypershift-aws-create-chain.yaml
  • ci-operator/step-registry/hypershift/aws/install-private/OWNERS
  • ci-operator/step-registry/hypershift/aws/install-private/config/OWNERS
  • ci-operator/step-registry/hypershift/aws/install-private/config/hypershift-aws-install-private-config-commands.sh
  • ci-operator/step-registry/hypershift/aws/install-private/config/hypershift-aws-install-private-config-ref.metadata.json
  • ci-operator/step-registry/hypershift/aws/install-private/config/hypershift-aws-install-private-config-ref.yaml
  • ci-operator/step-registry/hypershift/aws/install-private/hypershift-aws-install-private-chain.metadata.json
  • ci-operator/step-registry/hypershift/aws/install-private/hypershift-aws-install-private-chain.yaml
  • ci-operator/step-registry/hypershift/aws/install-private/hypershift-aws-install-private-commands.sh
  • ci-operator/step-registry/hypershift/aws/install-private/hypershift-aws-install-private-ref.metadata.json
  • ci-operator/step-registry/hypershift/aws/install-private/hypershift-aws-install-private-ref.yaml
  • ci-operator/step-registry/hypershift/aws/metadata/OWNERS
  • ci-operator/step-registry/hypershift/aws/metadata/hypershift-aws-metadata-commands.sh
  • ci-operator/step-registry/hypershift/aws/metadata/hypershift-aws-metadata-ref.metadata.json
  • ci-operator/step-registry/hypershift/aws/metadata/hypershift-aws-metadata-ref.yaml
  • ci-operator/step-registry/hypershift/aws/private-deprovision/OWNERS
  • ci-operator/step-registry/hypershift/aws/private-deprovision/hypershift-aws-private-deprovision-chain.metadata.json
  • ci-operator/step-registry/hypershift/aws/private-deprovision/hypershift-aws-private-deprovision-chain.yaml
  • ci-operator/step-registry/hypershift/aws/private-provision/OWNERS
  • ci-operator/step-registry/hypershift/aws/private-provision/hypershift-aws-private-provision-chain.metadata.json
  • ci-operator/step-registry/hypershift/aws/private-provision/hypershift-aws-private-provision-chain.yaml
  • ci-operator/step-registry/hypershift/calico/OWNERS
  • ci-operator/step-registry/hypershift/calico/health-check/OWNERS
  • ci-operator/step-registry/hypershift/calico/health-check/hypershift-calico-health-check-commands.sh
  • ci-operator/step-registry/hypershift/calico/health-check/hypershift-calico-health-check-ref.metadata.json
  • ci-operator/step-registry/hypershift/calico/health-check/hypershift-calico-health-check-ref.yaml
  • ci-operator/step-registry/hypershift/calico/install/OWNERS
  • ci-operator/step-registry/hypershift/calico/install/hypershift-calico-install-commands.sh
  • ci-operator/step-registry/hypershift/calico/install/hypershift-calico-install-ref.metadata.json
  • ci-operator/step-registry/hypershift/calico/install/hypershift-calico-install-ref.yaml
  • ci-operator/step-registry/hypershift/disable-guest/OWNERS
  • ci-operator/step-registry/hypershift/disable-guest/hypershift-disable-guest-commands.sh
  • ci-operator/step-registry/hypershift/disable-guest/hypershift-disable-guest-ref.metadata.json
  • ci-operator/step-registry/hypershift/disable-guest/hypershift-disable-guest-ref.yaml
  • ci-operator/step-registry/hypershift/enable-guest/OWNERS
  • ci-operator/step-registry/hypershift/enable-guest/hypershift-enable-guest-commands.sh
  • ci-operator/step-registry/hypershift/enable-guest/hypershift-enable-guest-ref.metadata.json
  • ci-operator/step-registry/hypershift/enable-guest/hypershift-enable-guest-ref.yaml
  • ci-operator/step-registry/hypershift/enable-qe/OWNERS
  • ci-operator/step-registry/hypershift/enable-qe/catalogsource/OWNERS
  • ci-operator/step-registry/hypershift/enable-qe/catalogsource/hypershift-enable-qe-catalogsource-chain.metadata.json
  • ci-operator/step-registry/hypershift/enable-qe/catalogsource/hypershift-enable-qe-catalogsource-chain.yaml
  • ci-operator/step-registry/hypershift/enable-qe/catalogsource/hypershift-enable-qe-catalogsource-commands.sh
  • ci-operator/step-registry/hypershift/enable-qe/catalogsource/hypershift-enable-qe-catalogsource-ref.metadata.json
  • ci-operator/step-registry/hypershift/enable-qe/catalogsource/hypershift-enable-qe-catalogsource-ref.yaml
  • ci-operator/step-registry/hypershift/enable-qe/pull-secret/OWNERS
  • ci-operator/step-registry/hypershift/enable-qe/pull-secret/hypershift-enable-qe-pull-secret-commands.sh
  • ci-operator/step-registry/hypershift/enable-qe/pull-secret/hypershift-enable-qe-pull-secret-ref.metadata.json
  • ci-operator/step-registry/hypershift/enable-qe/pull-secret/hypershift-enable-qe-pull-secret-ref.yaml
  • ci-operator/step-registry/hypershift/kubevirt/baremetalds/conformance-calico/hypershift-kubevirt-baremetalds-conformance-calico-workflow.yaml
  • ci-operator/step-registry/hypershift/mce/agent/metal3/create/calico/hypershift-mce-agent-metal3-create-calico-chain.yaml

@@ -0,0 +1,90 @@
#!/bin/bash

set -xeuo pipefail

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win

Disable xtrace by default in this step script.

set -x is enabled globally, but repo guidance for step-registry command scripts is to default to set -euo pipefail and enable tracing only when actively debugging.

Suggested change
-set -xeuo pipefail
+set -euo pipefail

As per coding guidelines, ci-operator/step-registry/**/*-commands.sh should “Use default set -euo pipefail (without -x) … only enable -x when actively debugging.”

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
set -xeuo pipefail
set -euo pipefail
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/hypershift/calico/install/hypershift-calico-install-commands.sh`
at line 3, Remove the `-x` flag from the `set -xeuo pipefail` command at the
beginning of the hypershift-calico-install-commands.sh script. According to repo
guidance for step-registry command scripts, the default should be `set -euo
pipefail` without xtrace enabled, and tracing should only be enabled when
actively debugging. Change the set command to remove the `x` option while
keeping the `e`, `u`, `o pipefail` options intact.

Source: Coding guidelines

Comment on lines +106 to +109
if [[ $SKIP_HYPERSHIFT_PULL_SECRET_UPDATE == "true" ]]; then
echo "SKIP ....."
exit 0
fi

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Do not gate catalogsource creation on the pull-secret skip flag.

Line 106 uses SKIP_HYPERSHIFT_PULL_SECRET_UPDATE to exit this step entirely, but that flag is documented as pull-secret-specific. This can unintentionally skip QE catalogsource setup and break downstream conformance behavior.

Suggested fix
-if [[ $SKIP_HYPERSHIFT_PULL_SECRET_UPDATE == "true" ]]; then
-  echo "SKIP ....."
-  exit 0
-fi
+# This step manages QE catalogsource only; do not reuse pull-secret skip gating here.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/hypershift/enable-qe/catalogsource/hypershift-enable-qe-catalogsource-commands.sh`
around lines 106 - 109, Remove the conditional check starting at line 106 that
uses SKIP_HYPERSHIFT_PULL_SECRET_UPDATE to gate the entire catalogsource
creation step. Since this flag is documented as pull-secret-specific and not
intended to control catalogsource setup, the catalogsource creation logic should
execute independently regardless of the pull-secret skip flag status. Delete the
if statement checking SKIP_HYPERSHIFT_PULL_SECRET_UPDATE and the exit 0 call so
that the catalogsource setup proceeds unconditionally.

@openshift-ci

openshift-ci Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: mgencur
Once this PR has been reviewed and has the lgtm label, please assign anuragthehatter, sjenning for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@mgencur mgencur force-pushed the calico_4.22_migrate_steps branch from 1db0e91 to 864717a Compare June 23, 2026 11:53
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@mgencur: pj-rehearse could not automatically process this event because the request waited in queue for longer than 5 minutes. Use /pj-rehearse to trigger rehearsals manually.

@mgencur

mgencur commented Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

/pj-rehearse periodic-ci-openshift-hypershift-release-4.22-periodics-e2e-aws-conformance-calico-private

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@mgencur: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@mgencur: pj-rehearse could not automatically process this event because the request waited in queue for longer than 5 minutes. Use /pj-rehearse to trigger rehearsals manually.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
ci-operator/step-registry/hypershift/aws/install-private/hypershift-aws-install-private-commands.sh (1)

94-95: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Avoid unconditional xtrace in this step script.

set -x is enabled for normal execution; prefer default non-tracing mode and gate tracing behind an explicit debug flag.

As per coding guidelines, step-registry command scripts should default to set -euo pipefail (without -x) and only enable -x when actively debugging.

Suggested change
-# Hypershift install
-set -x
+# Hypershift install
+if [[ "${HYPERSHIFT_DEBUG_TRACE:-false}" == "true" ]]; then
+  set -x
+fi
 "${COMMAND[@]}"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/hypershift/aws/install-private/hypershift-aws-install-private-commands.sh`
around lines 94 - 95, Remove the unconditional `set -x` command from the
hypershift-aws-install-private-commands.sh script. Instead, ensure the script
uses the default flags `set -euo pipefail` without the `-x` flag for normal
execution. If tracing is needed for debugging, gate the `set -x` behind an
explicit debug flag or environment variable check so that xtrace is only enabled
when actively debugging, not during normal execution.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In
`@ci-operator/step-registry/hypershift/aws/install-private/hypershift-aws-install-private-commands.sh`:
- Around line 94-95: Remove the unconditional `set -x` command from the
hypershift-aws-install-private-commands.sh script. Instead, ensure the script
uses the default flags `set -euo pipefail` without the `-x` flag for normal
execution. If tracing is needed for debugging, gate the `set -x` behind an
explicit debug flag or environment variable check so that xtrace is only enabled
when actively debugging, not during normal execution.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 74b628f6-25c3-42cb-8f47-ed3674b05848

📥 Commits

Reviewing files that changed from the base of the PR and between e3d6d68 and 864717a.

📒 Files selected for processing (8)
  • ci-operator/step-registry/hypershift/aws/install-private/config/hypershift-aws-install-private-config-commands.sh
  • ci-operator/step-registry/hypershift/aws/install-private/hypershift-aws-install-private-commands.sh
  • ci-operator/step-registry/hypershift/aws/install-private/hypershift-aws-install-private-ref.yaml
  • ci-operator/step-registry/hypershift/aws/metadata/hypershift-aws-metadata-commands.sh
  • ci-operator/step-registry/hypershift/calico/health-check/hypershift-calico-health-check-commands.sh
  • ci-operator/step-registry/hypershift/disable-guest/hypershift-disable-guest-commands.sh
  • ci-operator/step-registry/hypershift/enable-guest/hypershift-enable-guest-commands.sh
  • ci-operator/step-registry/hypershift/enable-qe/catalogsource/hypershift-enable-qe-catalogsource-chain.yaml
🚧 Files skipped from review as they are similar to previous changes (5)
  • ci-operator/step-registry/hypershift/disable-guest/hypershift-disable-guest-commands.sh
  • ci-operator/step-registry/hypershift/enable-qe/catalogsource/hypershift-enable-qe-catalogsource-chain.yaml
  • ci-operator/step-registry/hypershift/calico/health-check/hypershift-calico-health-check-commands.sh
  • ci-operator/step-registry/hypershift/aws/metadata/hypershift-aws-metadata-commands.sh
  • ci-operator/step-registry/hypershift/aws/install-private/config/hypershift-aws-install-private-config-commands.sh

@openshift-ci

openshift-ci Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

@mgencur: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant