Use of Azure IPAM in non-default Azure clouds

[joshmue]

General Information

• Cilium version 1.9.6
• AKS with Kubernetes 1.19.9

I attempted to deploy Cilium with Azure IPAM onto an AKS cluster in the AzureChinaCloud which differs from the public Azure cloud in some details.
Ref: https://docs.microsoft.com/en-us/azure/active-directory/develop/authentication-national-cloud
Ref: https://docs.microsoft.com/en-us/azure/china/resources-developer-guide

When using Azure CNI and Cilium in chaining mode, these differences do not matter, but with Cilium Azure IPAM, they do.

When preparing, I found that the Cilium Helm chart does not allow passing azure.cloudName yet, even though this is a valid flag for the operator already.

In the operator itself, the cloudName is used to configure the Azure client authorizer to acquire tokens from the correct AAD.
However, when creating the resource clients themselves, NewSomethingClient(subscriptionId) is called which defaults to the public Azure endpoints. Using NewSomethingClientWithBaseURI(baseURI, subscriptionId) should also allow calls to other Azure clouds. The respective endpoints can be loaded from here.

If you want, I could take a look into generalizing the code.

[ungureanuvladvictor]

@joshmue -- I've put a draft PR to add support for custom clouds but I do not have any way to test it at the moment. Can you give it a spin?

[joshmue]

Sure! I'll provide feedback ASAP.

[joshmue]

The operator certainly does not fail anymore. Also, .status.azure.interfaces of the CiliumNode resources is populated and the IP's of scheduled Pods seem to match CiliumEndpoints etc.
Connecting to destinations outside of the cluster does not work yet, but I am not sure whether this is related to this issue. I will investigate further.

[ungureanuvladvictor]

@joshmue -- in this case I think the Azure API interactions are g2g, that was the main thing blocking supporting custom endpoint clouds.