trusted-execution-clusters: Introduce Azure tests#79393
Conversation
openshift-ci
Bot
added
the
do-not-merge/work-in-progress
|
Skipping CI for Draft Pull Request. |
WalkthroughAdds Azure CI for trusted-execution-clusters: a CI operator variant that builds an Azure test image (Rust + Azure CLI), registers an optional nested-podman integration test, provides a step-registry ref and script to run Azure integration tests, and updates OWNERS metadata. ChangesAzure Integration Test Configuration
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Suggested labels
🚥 Pre-merge checks | ✅ 12✅ Passed checks (12 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: Jakob-Naucke The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci
Bot
added
the
approved
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@ci-operator/config/trusted-execution-clusters/operator/trusted-execution-clusters-operator-main__azure.yaml`:
- Around line 31-61: This config is missing the required top-level promotion
section; add a promotion: block alongside releases/tests/build_root/images that
provides the promotion namespace and a tag_specification mapping image stream
tags to the release tags (e.g., a namespace for the release image push and a
tag_specification object that lists the image stream names/tags to promote), so
the ci-operator can promote built images into the release stream; update the
existing config to include this promotion block at top level (referencing
promotion and tag_specification to align with the releases/tests entries).
In
`@ci-operator/step-registry/trusted-execution-clusters/ref/azure-integration-test/trusted-execution-clusters-ref-azure-integration-test-commands.sh`:
- Around line 9-23: Wrap all secret-handling commands (the podman login
invocation, the az login invocation, and the secret reads that set
AZURE_SUBSCRIPTION_ID and TEST_IMAGE) with a saved/restore xtrace guard: capture
current tracing state, disable tracing (set +x) before reading files and
invoking podman/az, then restore the original tracing state afterward; update
the block around the podman login, the AZURE_SUBSCRIPTION_ID/TEST_IMAGE exports
and the az login to use this save/restore pattern so credentials written by
VIRT_PROVIDER/PLATFORM/REGISTRY/TEST_IMAGE/AZURE_SUBSCRIPTION_ID are never
printed when tracing is enabled upstream.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Enterprise
Run ID: 70a64ce8-0bb8-4e16-a449-f00bf55e1a68
⛔ Files ignored due to path filters (1)
ci-operator/jobs/trusted-execution-clusters/operator/trusted-execution-clusters-operator-main-presubmits.yamlis excluded by!ci-operator/jobs/**
📒 Files selected for processing (6)
ci-operator/config/trusted-execution-clusters/operator/trusted-execution-clusters-operator-main__azure.yamlci-operator/step-registry/trusted-execution-clusters/ref/OWNERSci-operator/step-registry/trusted-execution-clusters/ref/azure-integration-test/OWNERSci-operator/step-registry/trusted-execution-clusters/ref/azure-integration-test/trusted-execution-clusters-ref-azure-integration-test-commands.shci-operator/step-registry/trusted-execution-clusters/ref/azure-integration-test/trusted-execution-clusters-ref-azure-integration-test-ref.metadata.jsonci-operator/step-registry/trusted-execution-clusters/ref/azure-integration-test/trusted-execution-clusters-ref-azure-integration-test-ref.yaml
💤 Files with no reviewable changes (1)
- ci-operator/step-registry/trusted-execution-clusters/ref/OWNERS
| releases: | ||
| latest: | ||
| candidate: | ||
| architecture: amd64 | ||
| product: ocp | ||
| stream: nightly | ||
| version: "4.21" | ||
| resources: | ||
| '*': | ||
| limits: | ||
| memory: 4Gi | ||
| requests: | ||
| cpu: 100m | ||
| memory: 200Mi | ||
| tests: | ||
| - always_run: false | ||
| as: integration-test | ||
| capabilities: | ||
| - nested-podman | ||
| optional: true | ||
| skip_if_only_changed: ^(\.github|LICENSES|bundle|docs|examples)/|^(README\.md|\.gitignore)$ | ||
| steps: | ||
| cluster_profile: openshift-org-aws | ||
| test: | ||
| - ref: trusted-execution-clusters-ref-azure-integration-test | ||
| workflow: ipi-aws | ||
| zz_generated_metadata: | ||
| branch: main | ||
| org: trusted-execution-clusters | ||
| repo: operator | ||
| variant: azure |
There was a problem hiding this comment.
Add the missing promotion section to this ci-operator config.
This config defines build_root, images, tests, and releases, but it omits promotion, which is required for files under ci-operator/config/.
Suggested structure
releases:
latest:
candidate:
architecture: amd64
product: ocp
stream: nightly
version: "4.21"
+promotion:
+ to:
+ - namespace: trusted-execution-clusters
+ name: operatorAs per coding guidelines, "CI configuration files should define build_root, images, tests (referencing step-registry workflows), promotion, and releases sections in ci-operator/config///".
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| releases: | |
| latest: | |
| candidate: | |
| architecture: amd64 | |
| product: ocp | |
| stream: nightly | |
| version: "4.21" | |
| resources: | |
| '*': | |
| limits: | |
| memory: 4Gi | |
| requests: | |
| cpu: 100m | |
| memory: 200Mi | |
| tests: | |
| - always_run: false | |
| as: integration-test | |
| capabilities: | |
| - nested-podman | |
| optional: true | |
| skip_if_only_changed: ^(\.github|LICENSES|bundle|docs|examples)/|^(README\.md|\.gitignore)$ | |
| steps: | |
| cluster_profile: openshift-org-aws | |
| test: | |
| - ref: trusted-execution-clusters-ref-azure-integration-test | |
| workflow: ipi-aws | |
| zz_generated_metadata: | |
| branch: main | |
| org: trusted-execution-clusters | |
| repo: operator | |
| variant: azure | |
| releases: | |
| latest: | |
| candidate: | |
| architecture: amd64 | |
| product: ocp | |
| stream: nightly | |
| version: "4.21" | |
| promotion: | |
| to: | |
| - namespace: trusted-execution-clusters | |
| name: operator | |
| resources: | |
| '*': | |
| limits: | |
| memory: 4Gi | |
| requests: | |
| cpu: 100m | |
| memory: 200Mi | |
| tests: | |
| - always_run: false | |
| as: integration-test | |
| capabilities: | |
| - nested-podman | |
| optional: true | |
| skip_if_only_changed: ^(\.github|LICENSES|bundle|docs|examples)/|^(README\.md|\.gitignore)$ | |
| steps: | |
| cluster_profile: openshift-org-aws | |
| test: | |
| - ref: trusted-execution-clusters-ref-azure-integration-test | |
| workflow: ipi-aws | |
| zz_generated_metadata: | |
| branch: main | |
| org: trusted-execution-clusters | |
| repo: operator | |
| variant: azure |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@ci-operator/config/trusted-execution-clusters/operator/trusted-execution-clusters-operator-main__azure.yaml`
around lines 31 - 61, This config is missing the required top-level promotion
section; add a promotion: block alongside releases/tests/build_root/images that
provides the promotion namespace and a tag_specification mapping image stream
tags to the release tags (e.g., a namespace for the release image push and a
tag_specification object that lists the image stream names/tags to promote), so
the ci-operator can promote built images into the release stream; update the
existing config to include this promotion block at top level (referencing
promotion and tag_specification to align with the releases/tests entries).
| podman login -u "$(oc whoami)" -p "$(oc whoami -t)" \ | ||
| image-registry.openshift-image-registry.svc:5000 --tls-verify=false | ||
|
|
||
| AZURE_SUBSCRIPTION_ID=$(cat /tmp/secrets/azure/subscription-id) | ||
| TEST_IMAGE=$(cat /tmp/secrets/azure/test-image) | ||
| export AZURE_SUBSCRIPTION_ID TEST_IMAGE | ||
|
|
||
| export VIRT_PROVIDER=azure | ||
| export PLATFORM=openshift | ||
| export REGISTRY="${REGISTRY}" | ||
|
|
||
| az login --service-principal \ | ||
| -u "$(cat /tmp/secrets/azure/client-id)" \ | ||
| -p "$(cat /tmp/secrets/azure/client-secret)" \ | ||
| --tenant "$(cat /tmp/secrets/azure/tenant-id)" |
There was a problem hiding this comment.
Guard secret/token operations by saving/restoring tracing state.
Sensitive values are handled on Lines 9-23, but there is no explicit xtrace guard. Add set +x with state restore around these operations to prevent credential leaks in CI logs when tracing is enabled upstream.
Proposed hardening patch
echo "[INFO] Logging in to internal registry..."
-podman login -u "$(oc whoami)" -p "$(oc whoami -t)" \
- image-registry.openshift-image-registry.svc:5000 --tls-verify=false
+restore_xtrace=0
+[[ $- == *x* ]] && restore_xtrace=1 && set +x
+podman login -u "$(oc whoami)" -p "$(oc whoami -t)" \
+ image-registry.openshift-image-registry.svc:5000 --tls-verify=false
+(( restore_xtrace )) && set -x
-AZURE_SUBSCRIPTION_ID=$(cat /tmp/secrets/azure/subscription-id)
-TEST_IMAGE=$(cat /tmp/secrets/azure/test-image)
+restore_xtrace=0
+[[ $- == *x* ]] && restore_xtrace=1 && set +x
+AZURE_SUBSCRIPTION_ID=$(cat /tmp/secrets/azure/subscription-id)
+TEST_IMAGE=$(cat /tmp/secrets/azure/test-image)
+(( restore_xtrace )) && set -x
export AZURE_SUBSCRIPTION_ID TEST_IMAGE
@@
-az login --service-principal \
+restore_xtrace=0
+[[ $- == *x* ]] && restore_xtrace=1 && set +x
+az login --service-principal \
-u "$(cat /tmp/secrets/azure/client-id)" \
-p "$(cat /tmp/secrets/azure/client-secret)" \
--tenant "$(cat /tmp/secrets/azure/tenant-id)"
+(( restore_xtrace )) && set -xAs per coding guidelines, "Use set +x and save/restore tracing state when handling passwords, tokens, or sensitive URLs in step registry scripts to prevent credential leakage in CI logs".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@ci-operator/step-registry/trusted-execution-clusters/ref/azure-integration-test/trusted-execution-clusters-ref-azure-integration-test-commands.sh`
around lines 9 - 23, Wrap all secret-handling commands (the podman login
invocation, the az login invocation, and the secret reads that set
AZURE_SUBSCRIPTION_ID and TEST_IMAGE) with a saved/restore xtrace guard: capture
current tracing state, disable tracing (set +x) before reading files and
invoking podman/az, then restore the original tracing state afterward; update
the block around the podman login, the AZURE_SUBSCRIPTION_ID/TEST_IMAGE exports
and the az login to use this save/restore pattern so credentials written by
VIRT_PROVIDER/PLATFORM/REGISTRY/TEST_IMAGE/AZURE_SUBSCRIPTION_ID are never
printed when tracing is enabled upstream.
|
/test all |
|
/pj-rehearse |
|
@Jakob-Naucke: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
Use native OpenShift as Trustee platform, attest Azure VMs against it. Optional test. Signed-off-by: Jakob Naucke <jnaucke@redhat.com> Assisted-by: Claude Opus 4.6
|
/retest |
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@ci-operator/config/trusted-execution-clusters/operator/trusted-execution-clusters-operator-main__azure.yaml`:
- Around line 11-12: The top-level `images` key is using an invalid object shape
with an `items:` sub-key; change `images:` to be a direct list (array) instead
of `images: items:` so each image entry is a YAML list item (e.g. `-
dockerfile_literal: | ...`, ` from: ...`, ` inputs: ...`, ` to: ...`); remove
the `items:` wrapper and un-indent its entries so the ci-operator can validate
the `images` list properly.
- Around line 51-55: Update the CI job to use Azure-specific settings: change
the cluster_profile value from openshift-org-aws to openshift-org-azure and
change the workflow from ipi-aws to ipi-azure so the test ref
trusted-execution-clusters-ref-azure-integration-test runs with the Azure
credentials and VIRT_PROVIDER=azure as expected.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Enterprise
Run ID: bfc57765-2cfa-410c-a385-f5e534190bd3
⛔ Files ignored due to path filters (1)
ci-operator/jobs/trusted-execution-clusters/operator/trusted-execution-clusters-operator-main-presubmits.yamlis excluded by!ci-operator/jobs/**
📒 Files selected for processing (6)
ci-operator/config/trusted-execution-clusters/operator/trusted-execution-clusters-operator-main__azure.yamlci-operator/step-registry/trusted-execution-clusters/ref/OWNERSci-operator/step-registry/trusted-execution-clusters/ref/azure-integration-test/OWNERSci-operator/step-registry/trusted-execution-clusters/ref/azure-integration-test/trusted-execution-clusters-ref-azure-integration-test-commands.shci-operator/step-registry/trusted-execution-clusters/ref/azure-integration-test/trusted-execution-clusters-ref-azure-integration-test-ref.metadata.jsonci-operator/step-registry/trusted-execution-clusters/ref/azure-integration-test/trusted-execution-clusters-ref-azure-integration-test-ref.yaml
💤 Files with no reviewable changes (1)
- ci-operator/step-registry/trusted-execution-clusters/ref/OWNERS
✅ Files skipped from review due to trivial changes (1)
- ci-operator/step-registry/trusted-execution-clusters/ref/azure-integration-test/OWNERS
🚧 Files skipped from review as they are similar to previous changes (3)
- ci-operator/step-registry/trusted-execution-clusters/ref/azure-integration-test/trusted-execution-clusters-ref-azure-integration-test-ref.yaml
- ci-operator/step-registry/trusted-execution-clusters/ref/azure-integration-test/trusted-execution-clusters-ref-azure-integration-test-ref.metadata.json
- ci-operator/step-registry/trusted-execution-clusters/ref/azure-integration-test/trusted-execution-clusters-ref-azure-integration-test-commands.sh
| images: | ||
| items: |
There was a problem hiding this comment.
Invalid images structure: remove the items: key.
The ci-operator schema expects images: to be a direct list, not an object with an items: sub-key. This will likely cause config validation or parsing failures.
Proposed fix
images:
- items:
- - dockerfile_literal: |
+- dockerfile_literal: |Full corrected structure:
images:
- dockerfile_literal: |
FROM nested-podman
...
from: nested-podman
inputs:
src:
paths:
- destination_dir: src
source_path: /go/src/github.com/trusted-execution-clusters/operator
to: azure-test-image🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@ci-operator/config/trusted-execution-clusters/operator/trusted-execution-clusters-operator-main__azure.yaml`
around lines 11 - 12, The top-level `images` key is using an invalid object
shape with an `items:` sub-key; change `images:` to be a direct list (array)
instead of `images: items:` so each image entry is a YAML list item (e.g. `-
dockerfile_literal: | ...`, ` from: ...`, ` inputs: ...`, ` to: ...`); remove
the `items:` wrapper and un-indent its entries so the ci-operator can validate
the `images` list properly.
| steps: | ||
| cluster_profile: openshift-org-aws | ||
| test: | ||
| - ref: trusted-execution-clusters-ref-azure-integration-test | ||
| workflow: ipi-aws |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check if there are Azure cluster profiles or workflows available
echo "=== Available Azure cluster profiles ==="
fd -t d 'azure' ci-operator/config/jobs --max-depth 3 2>/dev/null || echo "No azure job configs found"
echo -e "\n=== Azure workflows in step-registry ==="
fd -t d 'azure' ci-operator/step-registry --max-depth 4 2>/dev/null | head -20
echo -e "\n=== Existing Azure cluster profiles ==="
ls -la cluster/test-deploy/azure* 2>/dev/null || fd -t d 'azure' cluster/ --max-depth 3 2>/dev/null | head -10Repository: openshift/release
Length of output: 1266
🏁 Script executed:
# Check the file structure and look for __azure variants
echo "=== File under review ==="
cat ci-operator/config/trusted-execution-clusters/operator/trusted-execution-clusters-operator-main__azure.yaml
echo -e "\n=== Other __azure variants in same directory ==="
ls -la ci-operator/config/trusted-execution-clusters/operator/ | grep __Repository: openshift/release
Length of output: 1888
🏁 Script executed:
# Look for the test reference definition
echo "=== Search for the test reference ==="
rg 'trusted-execution-clusters-ref-azure-integration-test' --type yaml -A 5 -B 2Repository: openshift/release
Length of output: 2949
🏁 Script executed:
# Check for Azure cluster profile definitions
echo "=== Search for Azure cluster profiles ==="
rg 'openshift-org-azure' --type yamlRepository: openshift/release
Length of output: 50375
🏁 Script executed:
# Look at similar Azure test configurations
echo "=== Search for other Azure integration tests ==="
fd -t f '*.yaml' ci-operator/config | xargs grep -l 'azure.*integration' 2>/dev/null | head -5Repository: openshift/release
Length of output: 414
🏁 Script executed:
cat ci-operator/step-registry/trusted-execution-clusters/ref/azure-integration-test/trusted-execution-clusters-ref-azure-integration-test-commands.shRepository: openshift/release
Length of output: 917
🏁 Script executed:
# Check for similar test configs with cluster/workflow mismatch
rg 'cluster_profile:.*aws' ci-operator/config/trusted-execution-clusters/ -A 3 -B 1Repository: openshift/release
Length of output: 746
🏁 Script executed:
# Check CI naming convention violations in this directory
ls -la ci-operator/config/trusted-execution-clusters/operator/Repository: openshift/release
Length of output: 426
Fix cluster profile and workflow to match Azure integration test requirements.
The test trusted-execution-clusters-ref-azure-integration-test explicitly uses Azure credentials (subscription-id, client-id, client-secret, tenant-id) and runs az login with a service principal, setting VIRT_PROVIDER=azure. However, the configuration uses cluster_profile: openshift-org-aws and workflow: ipi-aws. This will cause the test to fail. Change both to use the Azure equivalents: cluster_profile: openshift-org-azure and workflow: ipi-azure.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@ci-operator/config/trusted-execution-clusters/operator/trusted-execution-clusters-operator-main__azure.yaml`
around lines 51 - 55, Update the CI job to use Azure-specific settings: change
the cluster_profile value from openshift-org-aws to openshift-org-azure and
change the workflow from ipi-aws to ipi-azure so the test ref
trusted-execution-clusters-ref-azure-integration-test runs with the Azure
credentials and VIRT_PROVIDER=azure as expected.
|
[REHEARSALNOTIFIER]
Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
|
@Jakob-Naucke: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
removed buggy line in containerfile looks like it had not been updated yet? |
1 participant
Use native OpenShift as Trustee platform, attest Azure VMs against it. Optional test.
Summary
This PR adds Azure integration testing support to the OpenShift CI configuration for the trusted-execution-clusters operator repository. It introduces an Azure-specific CI variant and a new step-registry ref that runs Azure-focused integration tests against a native OpenShift Trustee platform.
What changed (practical terms)
CI configuration: Adds a new CI operator config (trusted-execution-clusters-operator-main__azure.yaml) that registers an Azure variant for the operator repo. The variant:
Step-registry and test steps: Adds a new ref under ci-operator/step-registry/trusted-execution-clusters/ref/azure-integration-test:
OWNERS updates: Updates approvers/reviewers for the step-registry refs, removing
fangge1212andyalzhangand ensuringalicefrandJakob-Nauckeare listed as approvers/reviewers where applicable.Practical impact