CNTRLPLANE-3943: feat(hypershift-azure-deprovision): add --force flag and reduce grace period

[bryan-cox]

What this PR does / why we need it:

Integrates the --force flag from openshift/hypershift#9134 into the periodic Azure deprovisioner job (periodic-ci-openshift-hypershift-main-azure-deprovision-azure-deprovision).

When the grace period expires, --force strips all finalizers from child resources bottom-up (CAPI machines, clusters, HCP, deployments, NodePools, namespace) and continues with best-effort infrastructure cleanup.

Also reduces --cluster-grace-period from 40m to 20m.

Why this fixes the problem:

The Azure CI deprovisioner was failing to clean up stuck HostedClusters because:

1. The 7-layer finalizer chain gets blocked when Azure API calls fail
2. Processing 27+ stuck HCs serially at 40m/cluster exhausted the 4h Prow budget before any meaningful cleanup occurred

With --force and a reduced 20m grace period, stuck HCs are force-cleaned within the job budget.

Which issue(s) this PR fixes:

Fixes CNTRLPLANE-3943

Special notes for your reviewer:

• The --force flag and implementation logic were already merged in openshift/hypershift#9134
• This PR only integrates that flag into the CI deprovisioner script
• No changes to step-registry YAML are needed (no new env vars)
• Only one file changed: ci-operator/step-registry/hypershift/azure/deprovision/hypershift-azure-deprovision-commands.sh

Summary by CodeRabbit

• Updates the periodic Azure HostedCluster deprovisioner to use --force after the grace period expires.
• Reduces --cluster-grace-period from 40 to 20 minutes.
• Enables bottom-up finalizer removal and best-effort infrastructure cleanup within the 4-hour Prow budget.

[openshift-ci-robot]

@bryan-cox: This pull request references CNTRLPLANE-3943 which is a valid jira issue.

Details

In response to this:

What this PR does / why we need it:

Integrates the --force flag from openshift/hypershift#9134 into the periodic Azure deprovisioner job (periodic-ci-openshift-hypershift-main-azure-deprovision-azure-deprovision).

When the grace period expires, --force strips all finalizers from child resources bottom-up (CAPI machines, clusters, HCP, deployments, NodePools, namespace) and continues with best-effort infrastructure cleanup.

Also reduces --cluster-grace-period from 40m to 20m.

Why this fixes the problem:

The Azure CI deprovisioner was failing to clean up stuck HostedClusters because:

1. The 7-layer finalizer chain gets blocked when Azure API calls fail
2. Processing 27+ stuck HCs serially at 40m/cluster exhausted the 4h Prow budget before any meaningful cleanup occurred

With --force and a reduced 20m grace period, stuck HCs are force-cleaned within the job budget.

Which issue(s) this PR fixes:

Fixes CNTRLPLANE-3943

Special notes for your reviewer:

• The --force flag and implementation logic were already merged in openshift/hypershift#9134
• This PR only integrates that flag into the CI deprovisioner script
• No changes to step-registry YAML are needed (no new env vars)
• Only one file changed: ci-operator/step-registry/hypershift/azure/deprovision/hypershift-azure-deprovision-commands.sh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bryan-cox

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• ci-operator/step-registry/hypershift/azure/deprovision/OWNERS [bryan-cox]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 53dd3af5-331d-400b-87c4-55a7cc42cb30

📥 Commits

Reviewing files that changed from the base of the PR and between bdc4056 and 637bdc8.

📒 Files selected for processing (1)

• ci-operator/step-registry/hypershift/azure/deprovision/hypershift-azure-deprovision-commands.sh

---

Walkthrough

The Azure HostedCluster deprovision command now uses a 20-minute grace period and forces destruction.

Changes

HostedCluster deprovision

Layer / File(s)	Summary
Update HostedCluster destruction command ci-operator/step-registry/hypershift/azure/deprovision/hypershift-azure-deprovision-commands.sh	The command changes the grace period from 40 minutes to 20 minutes and adds --force. It retains failure counting.

Estimated code review effort: 1 (Trivial) | ~3 minutes

🚥 Pre-merge checks | ✅ 15

✅ Passed checks (15 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the added --force flag and the reduced grace period, which are the main changes in the pull request.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	The PR changes only a shell deprovisioner script; it adds no Ginkgo tests or test titles, so stable test-name requirements do not apply.
Test Structure And Quality	✅ Passed	The PR changes only one shell script and adds no Ginkgo test code, so the listed Ginkgo quality requirements are not applicable.
Microshift Test Compatibility	✅ Passed	The pull request changes only a shell script and adds no Ginkgo e2e tests, so MicroShift test compatibility does not apply.
Single Node Openshift (Sno) Test Compatibility	✅ Passed	The PR changes only a shell deprovisioner script; it adds no Ginkgo e2e tests or other tests requiring SNO compatibility review.
Topology-Aware Scheduling Compatibility	✅ Passed	The patch changes only a shell command's grace period and adds --force; it adds no manifests, operators, controllers, replicas, affinities, selectors, tolerations, or topology constraints.
Ote Binary Stdout Contract	✅ Passed	The PR changes only a Bash deprovisioner script; it adds no OTE binary or process-level test-suite code covered by this stdout contract.
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	The PR changes only a shell deprovisioner command; it adds no Ginkgo e2e tests or test networking and introduces no IPv4 or external-connectivity assumptions.
No-Weak-Crypto	✅ Passed	The change only adds Hypershift's --force flag and changes the grace period to 20m; no weak crypto, custom crypto, or secret comparison is introduced.
Container-Privileges	✅ Passed	The PR changes only --cluster-grace-period and adds --force; no privileged, host namespace, SYS_ADMIN, root, or allowPrivilegeEscalation setting appears in the patch or related step manifest.
No-Sensitive-Data-In-Logs	✅ Passed	The PR adds only --force and changes 40m to 20m; it adds no logging. Existing Azure credential handling disables tracing and suppresses az login output.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[bryan-cox]

/pj-rehearse ?

[openshift-merge-bot]

@bryan-cox: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[openshift-merge-bot]

[REHEARSALNOTIFIER]
@bryan-cox: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name	Repo	Type	Reason
periodic-ci-openshift-hypershift-main-azure-deprovision-azure-deprovision	N/A	periodic	Registry content changed

Prior to this PR being merged, you will need to either run and acknowledge or opt to skip these rehearsals.

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

[openshift-merge-bot]

@bryan-cox: job(s): ? either don't exist or were not found to be affected, and cannot be rehearsed

[bryan-cox]

/pj-rehearse periodic-ci-openshift-hypershift-main-azure-deprovision-azure-deprovision

[openshift-merge-bot]

@bryan-cox: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[openshift-ci]

@bryan-cox: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[bryan-cox]

/pj-rehearse ack

Rehearsal job rehearse-82817-periodic-ci-openshift-hypershift-main-azure-deprovision-azure-deprovision/2084298838499135488 passed successfully. Step-level build log confirms all 3 phases executed with the updated flags (--cluster-grace-period 20m --force):

=== Phase 1: HostedCluster Pruner (budget: 14400s) ===
No stale HostedClusters found.
=== Phase 2: Orphaned Resource Group Sweep ===
Deleting CI resource groups created before 2026-08-03T11:26:14Z ...
Phase 2: initiated deletion of 0 resource group(s).
Phase 2: complete.
=== Phase 3: Orphaned DNS Record Sweep ===
Found 4 active CI cluster prefix(es): 84369ce43a 7362ac055e e2a06ce698 514057a031
Sweeping DNS zone: hcp-sm-azure.azure.devcluster.openshift.com
Deleted 0 orphaned record(s) from hcp-sm-azure.azure.devcluster.openshift.com.
Sweeping DNS zone: sm.hcp-sm-azure.azure.devcluster.openshift.com
Deleted 0 orphaned record(s) from sm.hcp-sm-azure.azure.devcluster.openshift.com.
Phase 3: complete.
Azure deprovisioner finished successfully.

No stale resources existed at the time of the run, so --force was accepted but not exercised. The --force codepath (finalizer stripping) was already validated in openshift/hypershift#9134.

[openshift-merge-bot]

@bryan-cox: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.