feat(aro-hcp): lease ARM helper identities

[roivaz]

https://redhat.atlassian.net/browse/AROSLSRE-1679

Why

Concurrent ARO HCP E2E environments currently share the ARM helper application and therefore aggregate Microsoft Graph CheckAccess throttling across otherwise independent deployments. Azure/ARO-HCP#6376 created a 40-member identity pool, and #83011 added those identities to Boskos; the affected jobs can now lease dedicated identities.

What

• Request two resources through one count: 2 lease for DEV E2E, upgrade, HyperShift, and CAPZ provisioning paths.
• Parse and validate both whitespace-separated resource names in the two temporary release-side provision scripts.
• Use the first identity for the existing backend/shared ARM helper override and reserve the second for the upcoming dedicated Clusters Service wiring.
• Keep provision healthchecks lease-free.

The duplicate script changes are intentional while #82572 migrates these steps to the shared ARO-HCP helper. If that PR merges first, this PR will be rebased and the obsolete script copies removed.

Testing

• make registry-metadata
• make ci-operator-checkconfig
• Shell syntax checks for both provision scripts
• git diff --check
• Focused valid, wrong-count, duplicate, and unknown-resource lease tests

make validate-step-registry is currently blocked by the repository target passing the removed --prow-config flag to the latest ci-operator-configresolver image; the other config and registry validations pass.

Summary by CodeRabbit

• Updates ARO HCP CI jobs for DEV E2E, upgrade E2E, HyperShift, and CAPZ provisioning to lease two ARM helper identities.
• Validates both leased resource names in the provisioning scripts.
• Uses the first identity for ARM helper overrides and reserves the second for future Clusters Service integration.
• Keeps provision healthcheck jobs lease-free.
• Validation passed for configuration checks, registry metadata, job definitions, shell syntax, and focused lease tests. make validate-step-registry remains blocked by an obsolete --prow-config flag.

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[coderabbitai]

Walkthrough

Changes

ARO HCP ARM helper leases

Layer / File(s)	Summary
Configure ARM helper leases ci-operator/config/Azure/ARO-HCP/Azure-ARO-HCP-main__capz-e2e.yaml, ci-operator/step-registry/aro-hcp/*/*yaml	ARO HCP jobs and workflows request two aro-hcp-arm-helper-sp-dev resources through LEASED_ARM_HELPER_SP.
Apply validated lease data ci-operator/step-registry/aro-hcp/provision/*/*sh	Provisioning scripts validate both leases, read their metadata, and write the first lease’s client ID and certificate name to deployment overrides. The second lease remains reserved.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant AROHCPWorkflow
  participant ProvisioningScript
  participant ARMHelperPool
  participant ConfigOverride
  AROHCPWorkflow->>ProvisioningScript: Set LEASED_ARM_HELPER_SP with two resource names
  ProvisioningScript->>ARMHelperPool: Read clientId, principalId, and certName
  ARMHelperPool-->>ProvisioningScript: Return lease metadata
  ProvisioningScript->>ConfigOverride: Write first lease client ID and certificate name

Loading

Possibly related PRs

• openshift/release#83011: Introduces the aro-hcp-arm-helper-sp-dev Boskos resource used by these CI leases.

🚥 Pre-merge checks | ✅ 15

✅ Passed checks (15 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	The PR changes only YAML lease configuration and shell provisioning scripts; no Ginkgo test files or It/Describe/Context/When titles were added or changed.
Test Structure And Quality	✅ Passed	The PR changes only seven YAML/shell files. No test files or Ginkgo constructs were added or modified, so the Ginkgo test-quality requirements are not applicable.
Microshift Test Compatibility	✅ Passed	The PR changes only five YAML lease configurations and two shell scripts; it adds no Ginkgo tests, OpenShift API references, or MicroShift-sensitive test assumptions.
Single Node Openshift (Sno) Test Compatibility	✅ Passed	The PR adds only CI lease and provisioning YAML/shell changes; it adds no Ginkgo e2e tests or topology-sensitive test code, so SNO compatibility checks are not applicable.
Topology-Aware Scheduling Compatibility	✅ Passed	The diff changes CI lease declarations and shell-based ARM helper overrides only; it adds no manifests, controllers, replicas, affinity, topology spread, node selectors, tolerations, or PDBs.
Ote Binary Stdout Contract	✅ Passed	The PR changes only YAML and Bash lease/provisioning files; it adds no OTE binary code or process-level stdout/logging changes.
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	The patch adds no Ginkgo tests or new network code; it only adds lease entries and ARM helper provisioning logic in YAML and shell files.
No-Weak-Crypto	✅ Passed	The PR diff adds lease configuration and yq-based ARM-helper ID/certificate handling only; searches found no MD5, SHA1, DES, RC4, Blowfish, ECB, crypto APIs, or secret comparisons.
Container-Privileges	✅ Passed	The PR adds lease entries and shell validation only; the added lines contain no privileged:true, host PID/network/IPC, SYS_ADMIN, root, or allowPrivilegeEscalation settings.
No-Sensitive-Data-In-Logs	✅ Passed	The added logs print lease resource names only; client IDs and certificate names are identifiers. No password, token, API key, private key, PII, or customer data is logged.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: leasing ARM helper identities for ARO HCP.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[roivaz]

The Boskos resource type and generated 40-resource inventory have been split into #83011 so they can roll out before this PR requests the leases. Once #83011 merges, I will rebase this PR and drop the duplicate Boskos changes before running rehearsals.

[roivaz]

Rebasing onto current main now that #83011 has merged. The rebased diff drops core-services/prow/02_config/generate-boskos.py and _boskos.yaml; only the seven job lease and provision-script files remain. Local shell syntax, git diff --check, registry metadata generation, and make ci-operator-checkconfig pass. I will force-push with lease because the rebase rewrites the single PR commit.

[coderabbitai]

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

[openshift-merge-bot]

[REHEARSALNOTIFIER]
@roivaz: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name	Repo	Type	Reason
pull-ci-openshift-hypershift-main-e2e-aro-hcp	openshift/hypershift	presubmit	Registry content changed
pull-ci-openshift-hypershift-release-5.1-e2e-aro-hcp	openshift/hypershift	presubmit	Registry content changed
pull-ci-openshift-hypershift-release-5.0-e2e-aro-hcp	openshift/hypershift	presubmit	Registry content changed
pull-ci-openshift-hypershift-release-4.23-e2e-aro-hcp	openshift/hypershift	presubmit	Registry content changed
pull-ci-Azure-ARO-HCP-main-capz-e2e-dev	Azure/ARO-HCP	presubmit	Ci-operator config changed
pull-ci-Azure-ARO-HCP-main-e2e-parallel	Azure/ARO-HCP	presubmit	Registry content changed
pull-ci-Azure-ARO-HCP-main-e2e-parallel-inplace-upgrade	Azure/ARO-HCP	presubmit	Registry content changed
pull-ci-Azure-ARO-HCP-main-upgrade-e2e-parallel	Azure/ARO-HCP	presubmit	Registry content changed
periodic-ci-Azure-ARO-HCP-main-periodic-healthcheck-provision-westus3	N/A	periodic	Registry content changed
periodic-ci-Azure-ARO-HCP-main-periodic-healthcheck-provision-centralus	N/A	periodic	Registry content changed
periodic-ci-Azure-ARO-HCP-main-periodic-healthcheck-provision-canadacentral	N/A	periodic	Registry content changed

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

[roivaz]

/pj-rehearse pull-ci-Azure-ARO-HCP-main-e2e-parallel

[openshift-merge-bot]

@roivaz: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[roivaz]

/hold

[roivaz]

/hold cancel

[roivaz]

Rehearsal failed due to an alert - a knownIssue.

/pj-rehearse ack

[openshift-merge-bot]

@roivaz: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[mmazur]

/lgtm

[raelga]

/lgtm
/approve
/retest

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mmazur, raelga, roivaz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• ci-operator/config/Azure/ARO-HCP/OWNERS [mmazur,raelga,roivaz]
• ci-operator/step-registry/aro-hcp/hypershift-deploy/OWNERS [mmazur,raelga,roivaz]
• ci-operator/step-registry/aro-hcp/local-e2e/OWNERS [mmazur,raelga,roivaz]
• ci-operator/step-registry/aro-hcp/local-e2e-upgrade/OWNERS [mmazur,raelga,roivaz]
• ci-operator/step-registry/aro-hcp/provision/environment/OWNERS [mmazur,raelga,roivaz]
• ci-operator/step-registry/aro-hcp/provision/from-main/OWNERS [mmazur,raelga,roivaz]
• ci-operator/step-registry/aro-hcp/upgrade-e2e/OWNERS [mmazur,raelga,roivaz]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

@roivaz: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/rehearse/Azure/ARO-HCP/main/e2e-parallel	2d4e6ac	link	unknown	/pj-rehearse pull-ci-Azure-ARO-HCP-main-e2e-parallel

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.