AKS: after a pulumi refresh it wants to replace the cluster

[0x53A]

I created an AKS cluster. Then I ran pulumi refresh.

Now on the next pulumi up it wants to replace the whole cluster:

 +-  ├─ azure:containerservice:KubernetesCluster                                   psd-mes-aks                                                                                         replace     [diff: ~agentPoolProfiles]

but when I look at the diff, there aren't any actual changes. In the output, both agentPoolProfiles are exactly the same.

This is the agentPoolProfiles in my script

    agentPoolProfiles: [{
        name: "aksagentpool",
        count: 1,
        vmSize: "Standard_B2s",
        vnetSubnetId: aksSubnet.id,
        maxPods: 250,
        type: "VirtualMachineScaleSets"
    },
    ],

this is the diff output:

Do you want to perform this update? details
pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:dev::customerdeployments::pulumi:pulumi:Stack::customerdeployments-dev]
    ++azure:containerservice/kubernetesCluster:KubernetesCluster: (create-replacement)
        [id=/subscriptions/[[sub]]/resourcegroups/[[rg]]/providers/Microsoft.ContainerService/managedClusters/psd-mes-aks]
        [urn=urn:pulumi:dev::customerdeployments::azure:containerservice/kubernetesCluster:KubernetesCluster::psd-mes-aks]
        [provider=urn:pulumi:dev::customerdeployments::pulumi:providers:azure::default_1_2_0::db9647e2-64b5-4861-99ed-873fa33ff1e4]
      ~ agentPoolProfiles: [
          - [1]: {
                  - count            : 1
                  - enableAutoScaling: false
                  - fqdn             : "[[fqdn]].hcp.westeurope.azmk8s.io"
                  - maxCount         : 0
                  - maxPods          : 30
                  - minCount         : 0
                  - name             : "npwin"
                  - osDiskSizeGb     : 100
                  - osType           : "Windows"
                  - type             : "VirtualMachineScaleSets"
                  - vmSize           : "Standard_B2s"
                  - vnetSubnetId     : "/subscriptions/[[sub]]/resourceGroups/[[rg]]/providers/Microsoft.Network/virtualNetworks/psd-mes-aks-vnet9947b2c3/subnets/psd-mes-aks-subnet96b2b5c2"
                }
        ]
    +-azure:containerservice/kubernetesCluster:KubernetesCluster: (replace)
        [id=/subscriptions/[[sub]]/resourcegroups/[[rg]]/providers/Microsoft.ContainerService/managedClusters/psd-mes-aks]
        [urn=urn:pulumi:dev::customerdeployments::azure:containerservice/kubernetesCluster:KubernetesCluster::psd-mes-aks]
        [provider=urn:pulumi:dev::customerdeployments::pulumi:providers:azure::default_1_2_0::db9647e2-64b5-4861-99ed-873fa33ff1e4]
      ~ agentPoolProfiles: [
          - [1]: {
                  - count            : 1
                  - enableAutoScaling: false
                  - fqdn             : "[[fqdn]].hcp.westeurope.azmk8s.io"
                  - maxCount         : 0
                  - maxPods          : 30
                  - minCount         : 0
                  - name             : "npwin"
                  - osDiskSizeGb     : 100
                  - osType           : "Windows"
                  - type             : "VirtualMachineScaleSets"
                  - vmSize           : "Standard_B2s"
                  - vnetSubnetId     : "/subscriptions/[[sub]]/resourceGroups/[[rg]]/providers/Microsoft.Network/virtualNetworks/psd-mes-aks-vnet9947b2c3/subnets/psd-mes-aks-subnet96b2b5c2"
                }
        ]
    --outputs:--

[stack72]

Hi @0x53A

Please can you get the details view that shows what needs to change and post it here (minus any secrets!)

Paul

[0x53A]

oops, I had created a second Windows NodePool in the Portal manually (because Pulumi / TerraForm don't yet support that) and promptly forgot about it. Sorry about that, I'll go home.

Is there a way to tell the diff engine to ignore this?

(for reference, these are the blockers for windows node pool support afaik: hashicorp/terraform-provider-azurerm#3971
hashicorp/terraform-provider-azurerm#4001
)

[lukehoban]

Is there a way to tell the diff engine to ignore this?

Yes - you can use ignoreChanges: https://www.pulumi.com/docs/intro/concepts/programming-model/#ignorechanges

[0x53A]

Yes - you can use ignoreChanges

Neat, I added the flag, but now I get these errors:

C:\Progress\repos\customerdeployments\webapi-k8cluster>pulumi up
Previewing update (dev):

     Type                                         Name                     Plan     Info
     pulumi:pulumi:Stack                          customerdeployments-dev
     └─ azure:containerservice:KubernetesCluster  psd-mes-aks                       8 errors

Diagnostics:
  azure:containerservice:KubernetesCluster (psd-mes-aks):
    error: azure:containerservice/kubernetesCluster:KubernetesCluster resource 'psd-mes-aks' has a problem: expected agent_pool_profile.0.max_count to be in the range (1 - 100), got 0
    error: azure:containerservice/kubernetesCluster:KubernetesCluster resource 'psd-mes-aks' has a problem: "agent_pool_profile.0.dns_prefix": this field cannot be set
    error: azure:containerservice/kubernetesCluster:KubernetesCluster resource 'psd-mes-aks' has a problem: "agent_pool_profile.0.fqdn": this field cannot be set
    error: azure:containerservice/kubernetesCluster:KubernetesCluster resource 'psd-mes-aks' has a problem: expected agent_pool_profile.0.min_count to be in the range (1 - 100), got 0
    error: azure:containerservice/kubernetesCluster:KubernetesCluster resource 'psd-mes-aks' has a problem: expected agent_pool_profile.1.min_count to be in the range (1 - 100), got 0
    error: azure:containerservice/kubernetesCluster:KubernetesCluster resource 'psd-mes-aks' has a problem: expected agent_pool_profile.1.max_count to be in the range (1 - 100), got 0
    error: azure:containerservice/kubernetesCluster:KubernetesCluster resource 'psd-mes-aks' has a problem: "agent_pool_profile.1.dns_prefix": this field cannot be set
    error: azure:containerservice/kubernetesCluster:KubernetesCluster resource 'psd-mes-aks' has a problem: "agent_pool_profile.1.fqdn": this field cannot be set

I'd guess that it has filled in unset values (like minCount) with their defaults (0, ""). Do you have have a quick tip or should I open another issue / stackoverflow / slack?

Thank you.