Add Aro-hcp proposal #5605

serngawy · 2025-05-01T15:54:33Z

What type of PR is this?

/kind documentation

What this PR does / why we need it:

Proposal for ARO-HCP feature

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

TODOs:

squashed commits
[ X] includes documentation
adds unit tests
cherry-pick candidate

Release note:

None

codecov · 2025-05-01T16:05:01Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 52.84%. Comparing base (c3f0631) to head (2643a59).
Report is 99 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5605      +/-   ##
==========================================
- Coverage   53.28%   52.84%   -0.45%     
==========================================
  Files         272      278       +6     
  Lines       29537    29610      +73     
==========================================
- Hits        15739    15647      -92     
- Misses      12983    13146     +163     
- Partials      815      817       +2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

serngawy · 2025-05-05T14:38:48Z

@marek-veber

marek-veber · 2025-05-07T12:51:27Z

docs/proposals/20250425-aro-hcp.md

+
+ // Resource group name where the Aro-hcp will be attached to it.
+ ResourceGroup string `json:"resourceGroup,omitempty"`
+


IMHO we will also need the:

tenant_id,

subscription_id,

managed_resource_group_name (not only resource_group_name)
to create a cluster, isn't it?

yes, those info are provided at the azureclusteridentity

marek-veber · 2025-05-07T13:15:35Z

docs/proposals/20250425-aro-hcp.md

+// PlatformProfile represents the Azure platform configuration.
+type PlatformProfile struct {
+ // Azure subnet id
+ Subnet string `json:"subnet,omitempty"`


Why Subnet & SubnetRef ?
subnet id or cluster id or node pool id ? I'm not sure base on: https://github.com/Azure/ARO-HCP/blob/main/cluster-service/cluster-creation.md#creating-node-pools

Replicas & autorepare will be used from capi MachinePool?

subnetRef will be referring to VirtualNetworksSubnet CR by name.
yes, replicas defined at CAPI machinePool CR

marek-veber · 2025-05-07T17:59:32Z

docs/proposals/20250425-aro-hcp.md

+ ControlPlaneOperators  map[string]string `json:"controlPlaneOperators,omitempty"`
+
+
+ // DataPlaneOperators ref to Microsoft.ManagedIdentity/userAssignedIdentities
+ DataPlaneOperators     map[string]string `json:"dataPlaneOperators,omitempty"`
+
+
+ // ServiceManagedIdentity ref to Microsoft.ManagedIdentity/userAssignedIdentities
+ ServiceManagedIdentity string            `json:"serviceManagedIdentity,omitempty"`


What are the keys in those maps?
What about to replace maps with structures for it?

I updated the managed Identities to include all the required identities names

bryan-cox · 2025-05-09T19:24:07Z

docs/proposals/20250425-aro-hcp.md

+ ServiceManagedIdentity string            `json:"serviceManagedIdentity,omitempty"`
+}
+
+type ControlPlaneOperators struct {


Are we not planning to reuse the HyperShift API for this, rather than duplicating API efforts?

no, we will re-use the Hypershift API. The naming for CRD fields is a bit different but internally with API call will set the naming as here

bryan-cox · 2025-05-09T19:24:27Z

docs/proposals/20250425-aro-hcp.md

+	KmsManagedIdentities string `json:"kmsManagedIdentities,omitempty"`
+}
+
+type DataPlaneOperators struct {


Similar question as above, re: reusing HyperShift API

bryan-cox · 2025-05-09T19:27:37Z

docs/proposals/20250425-aro-hcp.md

+ ServiceManagedIdentity string            `json:"serviceManagedIdentity,omitempty"`
+}
+
+type ControlPlaneOperators struct {


Should we also list the exact role definitions each one of these will get?

not sure if I understand, The field desc mentioning the role definition "control-plane" and similar for every UAMI

Every control plane and data plane are assigned a specific Azure role definition(s) - CNTRLPLANE-171.

For example, the ingress managed identity is only assigned the "Azure Red Hat OpenShift Cluster Ingress Operator" role over the resource groups containing the VNET and the DNS Zone.

okay, that is fine we will use the same role naming and keys. Would you mention the ref for it from the Hypershift repo . I don't think the Jira url above is accessible for everyone.

The mapping of control plane to role definition to resource group does not reside from the HyperShift repo. I am not sure if it is available in some public fashion on the ARO side 🤔

I checked the hypershift repo, I don't think we need to set the Azure role definition, In CAPZ we will set the role key name similar to the example here ex; control-plane, cluster-api-azure, ingress, ..etc then hypershift will set the role definition. Hypershift has the role ref here

it is here in the ocm-cluster-service

FWIW, we do not set the role definitions in HyperShift. That is assumed to be done prior to creating the HostedCluster.

The HyperShift link you mentioned is related to the HyperShift CLI, which does set the role definitions but that is only for development and testing use.

hmm, so we need to have this defined some where yes/no ?
Also this is more like implementation details for the UserAssignedIdentity CR that will be created we will need to set the right spec info

I think it would be nice to be defined somewhere.

Maybe it goes with the UserAssignedIdentity CR; is that defined somewhere?

willie-yao · 2025-05-29T16:05:58Z

/assign

marek-veber · 2025-06-02T11:28:28Z

docs/proposals/20250425-aro-hcp.md

+	ControlPlaneManagedIdentities string `json:"controlPlaneOperatorsManagedIdentities,omitempty"`
+
+    // ClusterApiAzureManagedIdentities "cluster-api-azure" Microsoft.ManagedIdentity/userAssignedIdentities
+	ClusterApiAzureManagedIdentities string `json:clusterApiAzureManagedIdentities",omitempty"`


sometimes is here badly formatted json option:
json:clusterApiAzureManagedIdentities",omitempty" -> json:"clusterApiAzureManagedIdentities,omitempty"

The " should be moved after json: . There is such a typo multiple times.

Thanks fixed

k8s-ci-robot · 2025-06-02T20:21:50Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from willie-yao. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

marek-veber

identityRef - need to be *corev1.ObjectReference in go
and in yaml:

identityRef:
   kind: AzureClusterIdentity
   name: aro-identity
   namespace: default

marek-veber · 2025-06-02T23:01:55Z

docs/proposals/20250425-aro-hcp.md

+
+ // IdentityRef is a reference to an identity to be used when reconciling the aro control plane.
+ // If no identity is specified, the default identity for this controller will be used.
+ IdentityRef *infrav1.AzureClusterIdentity `json:"identityRef,omitempty"`


Suggested change

IdentityRef *infrav1.AzureClusterIdentity `json:"identityRef,omitempty"`

IdentityRef *corev1.ObjectReference `json:"identityRef,omitempty"`

marek-veber · 2025-06-02T23:04:21Z

docs/proposals/20250425-aro-hcp.md

+ identityRef:
+   name: aro-identity


Suggested change

identityRef:

name: aro-identity

identityRef:

kind: AzureClusterIdentity

name: aro-identity

namespace: default

marek-veber · 2025-06-02T23:28:20Z

docs/proposals/20250425-aro-hcp.md

+
+ // DiskStorageAccountType represents supported Azure storage account types.
+ // Available values are Premium_LRS, StandardSSD_LRS and Standard_LRS.
+ DiskStorageAccountType DiskStorageAccountType `json:"diskStorageAccountType,omitempty"`


The right type is IMHO enumeration compute.DiskStorageAccountTypes ... but make generate fails fit NPE :(

I'm suggesting:

Suggested change

DiskStorageAccountType DiskStorageAccountType `json:"diskStorageAccountType,omitempty"`

DiskStorageAccountType string `json:"diskStorageAccountType,omitempty"`

okay, with the kubebuilder tags it should be as string with enum validation tag.

marek-veber · 2025-06-02T23:29:43Z

docs/proposals/20250425-aro-hcp.md

+
+
+```yaml
+apiVersion: controlplane.cluster.x-k8s.io/v1alpha1


Suggested change

apiVersion: controlplane.cluster.x-k8s.io/v1alpha1

apiVersion: infrastructure.cluster.x-k8s.io/v1beta1

its a controlplane group but version should v1beta2

docs/proposals/20250425-aro-hcp.md

marek-veber · 2025-06-09T21:58:25Z

docs/proposals/20250425-aro-hcp.md

+    // DiskCsiDrivereManagedIdentities "disk-csi-driver" Microsoft.ManagedIdentity/userAssignedIdentities
+	DiskCsiDrivereManagedIdentities string `json:"diskCsiDrivereManagedIdentities,omitempty"`


Suggested change

// DiskCsiDrivereManagedIdentities "disk-csi-driver" Microsoft.ManagedIdentity/userAssignedIdentities

DiskCsiDrivereManagedIdentities string `json:"diskCsiDrivereManagedIdentities,omitempty"`

// DiskCsiDriverManagedIdentities "disk-csi-driver" Microsoft.ManagedIdentity/userAssignedIdentities

DiskCsiDriverManagedIdentities string `json:"diskCsiDriverManagedIdentities,omitempty"`

marek-veber · 2025-06-09T21:58:51Z

docs/proposals/20250425-aro-hcp.md

+	// DiskCsiDrivereManagedIdentities "disk-csi-driver" Microsoft.ManagedIdentity/userAssignedIdentities
+	DiskCsiDrivereManagedIdentities string `json:"diskCsiDrivereManagedIdentities,omitempty"`


linter's request:

Suggested change

// DiskCsiDrivereManagedIdentities "disk-csi-driver" Microsoft.ManagedIdentity/userAssignedIdentities

DiskCsiDrivereManagedIdentities string `json:"diskCsiDrivereManagedIdentities,omitempty"`

// DiskCsiDriverManagedIdentities "disk-csi-driver" Microsoft.ManagedIdentity/userAssignedIdentities

DiskCsiDriverManagedIdentities string `json:"diskCsiDriverManagedIdentities,omitempty"`

marek-veber · 2025-06-11T16:52:24Z

docs/proposals/20250425-aro-hcp.md

+
+
+ // ResourceGroup Ref name that is used to create the ResourceGroup CR. The ResourceGroupRef must be in the same namespace as the AroControlPlane and cannot be set with ResourceGroup.
+ ResourceGroupRef string `json:"resourceGroup,omitempty"`


Suggested change

ResourceGroupRef string `json:"resourceGroup,omitempty"`

ResourceGroupRef string `json:"resourceGroupRef,omitempty"`

marek-veber · 2025-06-11T16:54:39Z

docs/proposals/20250425-aro-hcp.md

+    // ClusterApiAzureManagedIdentities "cluster-api-azure" Microsoft.ManagedIdentity/userAssignedIdentities
+	ClusterApiAzureManagedIdentities string `json:"clusterApiAzureManagedIdentities,omitempty"`


Suggested change

// ClusterApiAzureManagedIdentities "cluster-api-azure" Microsoft.ManagedIdentity/userAssignedIdentities

ClusterApiAzureManagedIdentities string `json:"clusterApiAzureManagedIdentities,omitempty"`

// ClusterAPIAzureManagedIdentities "cluster-api-azure" Microsoft.ManagedIdentity/userAssignedIdentities

ClusterAPIAzureManagedIdentities string `json:"clusterApiAzureManagedIdentities,omitempty"`

marek-veber · 2025-06-11T16:56:23Z

docs/proposals/20250425-aro-hcp.md

+ // ApiURL is the url for the ARO-HCP openshift cluster api endPoint.
+ ApiURL string `json:"apiURL,omitempty"`


linter's reguest:

Suggested change

// ApiURL is the url for the ARO-HCP openshift cluster api endPoint.

ApiURL string `json:"apiURL,omitempty"`

// APIURL is the url for the ARO-HCP openshift cluster api endPoint.

APIURL string `json:"apiURL,omitempty"`

Signed-off-by: serngawy <serngawy@gmail.com>

github-project-automation bot moved this to Todo in CAPZ Planning May 1, 2025

github-project-automation bot added this to CAPZ Planning May 1, 2025

k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. kind/documentation Categorizes issue or PR as related to documentation. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels May 1, 2025

k8s-ci-robot requested review from mboersma and nojnhuh May 1, 2025 15:54

k8s-ci-robot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label May 1, 2025

serngawy force-pushed the aro-proposal branch 2 times, most recently from 22ad098 to 35b0abc Compare May 2, 2025 21:01

marek-veber reviewed May 7, 2025

View reviewed changes

serngawy force-pushed the aro-proposal branch from 35b0abc to 169d04e Compare May 7, 2025 20:35

bryan-cox reviewed May 9, 2025

View reviewed changes

k8s-ci-robot assigned willie-yao May 29, 2025

marek-veber reviewed Jun 2, 2025

View reviewed changes

serngawy force-pushed the aro-proposal branch from 169d04e to d9bc85b Compare June 2, 2025 20:21

marek-veber reviewed Jun 2, 2025

View reviewed changes

marek-veber mentioned this pull request Jun 3, 2025

ARO ... define structures #5671

Open

4 tasks

marek-veber reviewed Jun 3, 2025

View reviewed changes

serngawy force-pushed the aro-proposal branch from d9bc85b to 44837d6 Compare June 4, 2025 18:07

marek-veber suggested changes Jun 9, 2025

View reviewed changes

github-project-automation bot moved this from Todo to Wait-On-Author in CAPZ Planning Jun 9, 2025

marek-veber suggested changes Jun 11, 2025

View reviewed changes

Add Aro-hcp proposal

2643a59

Signed-off-by: serngawy <serngawy@gmail.com>

serngawy force-pushed the aro-proposal branch from 44837d6 to 2643a59 Compare June 11, 2025 21:22

mboersma added this to the v1.21 milestone Jun 12, 2025


		// Resource group name where the Aro-hcp will be attached to it.
		ResourceGroup string `json:"resourceGroup,omitempty"`

	IdentityRef *infrav1.AzureClusterIdentity `json:"identityRef,omitempty"`
	IdentityRef *corev1.ObjectReference `json:"identityRef,omitempty"`

	DiskStorageAccountType DiskStorageAccountType `json:"diskStorageAccountType,omitempty"`
	DiskStorageAccountType string `json:"diskStorageAccountType,omitempty"`

	apiVersion: controlplane.cluster.x-k8s.io/v1alpha1
	apiVersion: infrastructure.cluster.x-k8s.io/v1beta1

		// DiskCsiDrivereManagedIdentities "disk-csi-driver" Microsoft.ManagedIdentity/userAssignedIdentities
		DiskCsiDrivereManagedIdentities string `json:"diskCsiDrivereManagedIdentities,omitempty"`



		// ResourceGroup Ref name that is used to create the ResourceGroup CR. The ResourceGroupRef must be in the same namespace as the AroControlPlane and cannot be set with ResourceGroup.
		ResourceGroupRef string `json:"resourceGroup,omitempty"`

		// ClusterApiAzureManagedIdentities "cluster-api-azure" Microsoft.ManagedIdentity/userAssignedIdentities
		ClusterApiAzureManagedIdentities string `json:"clusterApiAzureManagedIdentities,omitempty"`

		// ApiURL is the url for the ARO-HCP openshift cluster api endPoint.
		ApiURL string `json:"apiURL,omitempty"`

Add Aro-hcp proposal #5605

Are you sure you want to change the base?

Add Aro-hcp proposal #5605

Conversation

serngawy commented May 1, 2025

Uh oh!

codecov bot commented May 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

serngawy commented May 5, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

willie-yao commented May 29, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

k8s-ci-robot commented Jun 2, 2025

Uh oh!

marek-veber left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

marek-veber Jun 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

codecov bot commented May 1, 2025 •

edited

Loading

marek-veber Jun 9, 2025 •

edited

Loading

marek-veber Jun 9, 2025 •

edited

Loading