Node pool update via managed cluster not allowed. Use per nodepool operations

[kim0]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

Terraform v0.12.0

• provider.azuread v0.3.1
• provider.azurerm v1.29.0
• provider.helm v0.9.1
• provider.kubernetes v1.7.0
• provider.random v2.1.2

Affected Resource(s)

• azurerm_kubernetes_cluster

Diff

      ~ agent_pool_profile {
          ~ count           = 3 -> 5
            fqdn            = "foo-bar.hcp.westeurope.azmk8s.io"
            max_pods        = 30
            name            = "fast"
            os_disk_size_gb = 100
            os_type         = "Linux"
            type            = "VirtualMachineScaleSets"
            vm_size         = "Standard_F4s_v2"
            vnet_subnet_id  = "/subscriptions/REDACTED/resourceGroups/foo/providers/Microsoft.Network/virtualNetworks/foovnet/subnets/k8snodes"
        }

Expected Behavior

TF scales the agent pool

Actual Behavior

Error: Error creating/updating Managed Kubernetes Cluster "dustydog" (Resource Group "dustydog"): containerservice.ManagedClustersClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="OperationNotAllowed" Message="Node pool update via managed cluster not allowed. Use per nodepool operations."

Steps to Reproduce

1. Create an aks cluster with 2 node pools
2. Change the size of a node pool within HCL
3. terraform apply

[kim0]

Hey @katbyte .. is this a well known issue? Any known work-arounds yet ? Thanks a lot for helping 👍

[lnovara]

Hi, I am having the same problem as @kim0.

This issue is making managing AKS clusters through Terraform practically impossibile.

@katbyte, any news about when this will be addressed?

[TVH7]

Just updated to 1.32 and got the same issue. No way of updating the agent pool min and max_count as I'm blocked by the error message.

containerservice.ManagedClustersClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="OperationNotAllowed" Message="Node pool update via managed cluster not allowed. Use per nodepool operations."

[kim0]

@TVH7 sometimes it's useful to use

  lifecycle {
    ignore_changes = ["agent_pool_profile"]
  }

and with the new provider, I will probably enable auto-scaling anyway!

[jluk]

Hey folks - AKS person here, catching up. The issue you are seeing happens when a cluster has multiple node pools enabled, thus actions like update/scale need to happen through the agent pool profile instead of the managed cluster for us to distinguish which node pool should be changed.

Silly question - does Terraform already support multiple node pools?

[alex-goncharov]

@jluk yes it does, the issue is that provider tries to update node pool with AKS api, while it should do so with node pool API

[djsly]

@titilambert this is the issue :) people are going crazy, lets try to fix tomorrow ? @alex-goncharov that's the issue indeed, @jluk yes, is does, we added the support to allow having hybrid windows/linux AKS clusters.

If no one is looking at this now, we will be working on this soon, as it is a blocker for us as well.

[davidack]

It's not just updates, this is affecting AKS cluster creation as well. Sometime in the past week the API was changed: I have Terraform code with 4 agent_pool_profile resources in my azure_kubernetes_cluster which worked fine last week to provision a new AKS cluster, but now results in this error:

Error: Error creating/updating Managed Kubernetes Cluster "foo" (Resource Group "foo-rg"): containerservice.ManagedClustersClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="AgentPoolUpdateViaManagedClusterNotSupported" Message="Node pool update via managed cluster not allowed when the cluster contains more than one node pool. Please use per node pool operations."

It looks like the work @titilambert is doing in #4001 will fix this in 1.33 of the azurerm provider by providing the new azurerm_kubernetes_cluster_agentpool resource, but until that is released, could whoever made this API change back it out? As it is, I am now unable to provision an AKS cluster with Terraform due to this problem.

[ghost]

This has been released in version 1.37.0 of the provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading. As an example:

provider "azurerm" {
    version = "~> 1.37.0"
}
# ... other configuration ...

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!