Skip to content

fix(aro-hcp): pin prod-rh e2e jobs to RH-tenant subscriptions - #82461

Merged
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
roivaz:fix-prod-periodics
Jul 27, 2026
Merged

fix(aro-hcp): pin prod-rh e2e jobs to RH-tenant subscriptions#82461
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
roivaz:fix-prod-periodics

Conversation

@roivaz

@roivaz roivaz commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Why

The PROD ARO-HCP e2e jobs that run against the RH tenant (VAULT_SECRET_PROFILE: prod-rh) mount only the RH-tenant cluster profile dir (/var/run/aro-hcp-prod-rh) but set no ALLOWED_SUBSCRIPTIONS. Since STG/PROD were migrated to the slot-manager leasing model and the prod pool now spans two tenants, slot-manager acquire could lease the MSFT Test-Tenant sub (ARO HCP E2E) from the prod pool. The acquire step's VerifyCustomerSubscriptionName then fails intermittently with:

no customer subscription name file matched slot subscription "XXXXXXXX" in /var/run/aro-hcp-prod-rh

because the Test-Tenant credentials are not mounted for these RH-only jobs.

What

Constrain ALLOWED_SUBSCRIPTIONS to the two RH-tenant prod subs on all six affected jobs (all run aro-hcp-persistent-e2earo-hcp-lease-acquire):

ALLOWED_SUBSCRIPTIONS: ARO HCP E2E Hosted Clusters - Prod - 00,ARO HCP E2E Hosted Clusters - Prod - 01
  • Presubmits (Azure-ARO-HCP-main.yaml): prod-e2e-parallel, -ocp-stable, -ocp-fast, -ocp-nightly
  • Periodics (Azure-ARO-HCP-main__periodic.yaml): prod-e2e-parallel, -ocp-nightly

The acquire step already prefers this allowlist (--allowed-subscriptions), so the Test-Tenant sub is no longer handed out to RH-only jobs.

The cross-tenant delete-expired-prod-00/01-resource-groups cleanup jobs are unaffected — they target a fixed CUSTOMER_SUBSCRIPTION via aro-hcp-deprovision-expired-resource-groups and do not lease a slot.

Summary by CodeRabbit

Restricts six production ARO-HCP e2e jobs—four presubmit and two periodic—to the two RH-tenant production subscriptions. This prevents slot-manager from leasing the MSFT Test-Tenant subscription for jobs that only mount RH-tenant credentials, avoiding intermittent subscription verification failures.

Cross-tenant cleanup jobs are unaffected, and configuration validation and job generation checks pass without generated-job changes.

The PROD e2e jobs that run against the RH tenant (VAULT_SECRET_PROFILE:
prod-rh) mount only the RH-tenant cluster profile dir
(/var/run/aro-hcp-prod-rh) but set no ALLOWED_SUBSCRIPTIONS. As a result
slot-manager acquire could lease the MSFT Test-Tenant sub ("ARO HCP E2E")
from the prod pool, and the acquire step's VerifyCustomerSubscriptionName
then fails with:

  no customer subscription name file matched slot subscription "..." in
  /var/run/aro-hcp-prod-rh

because the Test-Tenant sub's credentials are not mounted for these jobs.

Constrain ALLOWED_SUBSCRIPTIONS to the two RH-tenant prod subs on all six
affected jobs (presubmits prod-e2e-parallel, -ocp-stable, -ocp-fast,
-ocp-nightly and periodics prod-e2e-parallel, -ocp-nightly), all of which
run aro-hcp-persistent-e2e -> aro-hcp-lease-acquire. The cross-tenant
delete-expired-prod-00/01 cleanup jobs are unaffected (fixed
CUSTOMER_SUBSCRIPTION, no slot lease).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@openshift-ci
openshift-ci Bot requested review from deads2k and mmazur July 27, 2026 06:58
@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 27, 2026
@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Walkthrough

Production ARO-HCP E2E job configurations now set ALLOWED_SUBSCRIPTIONS to the two specified hosted-cluster production subscriptions across main and periodic pipelines.

Changes

ARO-HCP production subscription restrictions

Layer / File(s) Summary
Configure allowed subscriptions for production E2E jobs
ci-operator/config/Azure/ARO-HCP/Azure-ARO-HCP-main.yaml, ci-operator/config/Azure/ARO-HCP/Azure-ARO-HCP-main__periodic.yaml
The production parallel, stable, fast, and nightly E2E jobs define ALLOWED_SUBSCRIPTIONS with the specified “Prod - 00” and “Prod - 01” hosted-cluster subscriptions.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Possibly related PRs

Suggested reviewers: ugreener

🚥 Pre-merge checks | ✅ 15
✅ Passed checks (15 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed Diff only adds ALLOWED_SUBSCRIPTIONS env vars in ci-operator YAML; no Ginkgo test titles (It/Describe/Context/When) were changed.
Test Structure And Quality ✅ Passed The PR only edits two ci-operator YAML configs; no Ginkgo test files or test code are changed, so this check is not applicable.
Microshift Test Compatibility ✅ Passed Only ci-operator YAML env vars changed; no new Ginkgo tests, APIs, or MicroShift-sensitive code were added, so the check is not applicable.
Single Node Openshift (Sno) Test Compatibility ✅ Passed Only ci-operator config YAML changed; no Ginkgo tests were added or modified, so the SNO compatibility check is not applicable.
Topology-Aware Scheduling Compatibility ✅ Passed The PR only adds ALLOWED_SUBSCRIPTIONS env vars in ci-operator job configs; no topology, affinity, PDB, nodeSelector, or replica changes were introduced.
Ote Binary Stdout Contract ✅ Passed Only ci-operator YAML env additions changed; no process-level code or stdout writes were introduced.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed Only ci-operator YAML changed; no Ginkgo tests or network-using code were added, so the check is not applicable.
No-Weak-Crypto ✅ Passed Only YAML env var additions; no MD5/SHA1/DES/RC4/3DES/Blowfish/ECB, custom crypto, or secret/token comparisons found.
Container-Privileges ✅ Passed Only ALLOWED_SUBSCRIPTIONS env entries changed in the two ci-operator YAMLs; no privileged/root/network/IPC/SYS_ADMIN/allowPrivilegeEscalation fields were added.
No-Sensitive-Data-In-Logs ✅ Passed The diff only adds ALLOWED_SUBSCRIPTIONS env vars to CI job configs; no new log statements or sensitive fields were introduced.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately and concisely describes restricting PROD ARO-HCP e2e jobs to RH-tenant subscriptions.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

[REHEARSALNOTIFIER]
@roivaz: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name Repo Type Reason
pull-ci-Azure-ARO-HCP-main-prod-e2e-parallel Azure/ARO-HCP presubmit Ci-operator config changed
pull-ci-Azure-ARO-HCP-main-prod-e2e-parallel-ocp-fast Azure/ARO-HCP presubmit Ci-operator config changed
pull-ci-Azure-ARO-HCP-main-prod-e2e-parallel-ocp-nightly Azure/ARO-HCP presubmit Ci-operator config changed
pull-ci-Azure-ARO-HCP-main-prod-e2e-parallel-ocp-stable Azure/ARO-HCP presubmit Ci-operator config changed
periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel N/A periodic Ci-operator config changed
periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel-ocp-nightly N/A periodic Ci-operator config changed

Prior to this PR being merged, you will need to either run and acknowledge or opt to skip these rehearsals.

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@raelga raelga left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/appove

@openshift-ci

openshift-ci Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

@roivaz: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jul 27, 2026
@openshift-ci

openshift-ci Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: raelga, roivaz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@roivaz

roivaz commented Jul 27, 2026

Copy link
Copy Markdown
Contributor Author

/pj-rehearse ack

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@roivaz: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-merge-bot openshift-merge-bot Bot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Jul 27, 2026
@openshift-merge-bot
openshift-merge-bot Bot merged commit 339ec7a into openshift:main Jul 27, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. rehearsals-ack Signifies that rehearsal jobs have been acknowledged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants