[Bug]: Unable to upgrade KubernetesCluster version

[rbrunan]

Is there an existing issue for this?

• I have searched the existing issues

Affected Resource(s)

containerservice.azure.upbound.io - KubernetesCluster

Resource MRs required to reproduce the bug

containerservice.azure.upbound.io - KubernetesCluster

Steps to Reproduce

• Deploy a KubernetesCluster MR with networkProfile.0.networkPlugin: none and wait for the cluster to get ready.
• Update the field spec.forProvider.kubernetesVersion in the MR to upgrade the cluster version.
• Check the MR status.

What happened?

The cluster is not upgraded and the MR is falling with a AsyncUpdateFailure.

Relevant Error Output Snippet

Message: async update failed: failed to update the resource: [{0 updating Kubernetes Version for Managed Cluster ( Subscription: "xxxx"
Resource Group Name: "xxxx" Managed Cluster Name: "xxxxx"): managedclusters.ManagedClustersClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Origin al Error: Code="PropertyChangeNotAllowed" Message="Changing property 'networkProfile.podCIDR' from '10.244.0.0/16' to '' is not allowed." Target="networkProfile.podCIDR"  []}]                                                                                                                                      

Reason:                AsyncUpdateFailure                                                                                                              Status:                False                                                                                                                           
Type:                  LastAsyncOperation

Crossplane Version

1.14.3

Provider Version

0.41.0

Kubernetes Version

v1.27.1

Kubernetes Distribution

AKS

Additional Info

Test MR:

apiVersion: containerservice.azure.upbound.io/v1beta1
kind: KubernetesCluster
metadata:
  name: upgrade-test-2
spec:
  deletionPolicy: Delete
  forProvider:
    defaultNodePool:
    - enableAutoScaling: false
      name: default
      nodeCount: 1
      vmSize: Standard_D2s_v3
      vnetSubnetIdSelector:
        matchLabels:
          account-name: app-1
    identity:
    - type: SystemAssigned
    kubernetesVersion: "1.27"
    location: eastus2
    networkProfile:
    - dnsServiceIp: 10.253.0.10
      networkPlugin: none
      serviceCidr: 10.253.0.0/16
    oidcIssuerEnabled: true
    publicNetworkAccessEnabled: true
    resourceGroupNameSelector:
      matchLabels:
        account-name: app-1
    roleBasedAccessControlEnabled: true
    runCommandEnabled: true
    skuTier: Free
    tags:
      environment: dev
    workloadIdentityEnabled: true
  managementPolicies:
  - '*'

[rbrunan]

This seems to be related to the fact that the Azure API is returning an empty value for the podCidr if you are using networkPlugin: none:

az aks show --resource-group=$RG --name upgrade-test-li --query "networkProfile"  --subscription=$SUBSID

{
  "dnsServiceIp": "10.253.0.10",
  "ipFamilies": [
    "IPv4"
  ],
  "loadBalancerProfile": {
    "allocatedOutboundPorts": null,
    "effectiveOutboundIPs": [
      {
        "id": "..."
        "resourceGroup": "RG..."
      }
    ],
    "enableMultipleStandardLoadBalancers": null,
    "idleTimeoutInMinutes": null,
    "managedOutboundIPs": {
      "count": 1,
      "countIpv6": null
    },
    "outboundIPs": null,
    "outboundIpPrefixes": null
  },
  "loadBalancerSku": "Standard",
  "natGatewayProfile": null,
  "networkDataplane": null,
  "networkMode": null,
  "networkPlugin": "none",
  "networkPluginMode": null,
  "networkPolicy": null,
  "outboundType": "loadBalancer",
  "podCidr": null,
  "podCidrs": null,
  "serviceCidr": "10.253.0.0/16",
  "serviceCidrs": [
    "10.253.0.0/16"
  ]
}

[rbrunan]

I've seen the same behavior in terraform in versions below v3.90.0. But the current provider version used in this Crossplane provider is v3.57.0 AFAIK.
It's a considerable gap :(

[jeanduplessis]

@rbrunan FYI #497 (comment)

[rbrunan]

@rbrunan FYI #497 (comment)

Thank you @jeanduplessis, I can't agree more with that comment :)