kube_dashboard flapping for azurerm_kubernetes_cluster

[pst]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

Terraform: 0.12.28
Provider: 2.18.0

Affected Resource(s)

• azurerm_kubernetes_cluster

Terraform Configuration Files

https://github.com/kbst/terraform-kubestack/blob/07920b7e9aaec50b474c4071b4dc10ed01c67394/azurerm/_modules/aks/main.tf

Plan/Apply Output

Step #4 - "terraform plan": ------------------------------------------------------------------------
Step #4 - "terraform plan": 
Step #4 - "terraform plan": An execution plan has been generated and is shown below.
Step #4 - "terraform plan": Resource actions are indicated with the following symbols:
Step #4 - "terraform plan":   ~ update in-place
Step #4 - "terraform plan": 
Step #4 - "terraform plan": Terraform will perform the following actions:
Step #4 - "terraform plan": 
Step #4 - "terraform plan":   # module.aks_zero.module.cluster.azurerm_kubernetes_cluster.current will be updated in-place
Step #4 - "terraform plan":   ~ resource "azurerm_kubernetes_cluster" "current" {
Step #4 - "terraform plan":         [...]
Step #4 - "terraform plan": 
Step #4 - "terraform plan":       ~ addon_profile {
Step #4 - "terraform plan": 
Step #4 - "terraform plan":           - kube_dashboard {
Step #4 - "terraform plan":               - enabled = true -> null
Step #4 - "terraform plan":             }
Step #4 - "terraform plan": 
Step #4 - "terraform plan":             oms_agent {
Step #4 - "terraform plan":                 enabled                    = true
Step #4 - "terraform plan":                 log_analytics_workspace_id = "redacted"
Step #4 - "terraform plan":                 oms_agent_identity         = []
Step #4 - "terraform plan":             }
Step #4 - "terraform plan":         }
Step #4 - "terraform plan":     [...]
Step #4 - "terraform plan":     }
Step #4 - "terraform plan": 
Step #4 - "terraform plan": Plan: 0 to add, 1 to change, 0 to destroy.
Finished Step #4 - "terraform plan"
2020/07/13 12:50:34 Step Step #4 - "terraform plan" finished
Starting Step #5 - "terraform apply"
Step #5 - "terraform apply": Already have image: kbst-infra-automation:bootstrap
Step #5 - "terraform apply": module.aks_zero.module.cluster.azurerm_kubernetes_cluster.current: Modifying... [id=/subscriptions/803f877a-b0ff-49a7-afc7-eea135679b9a/resourcegroups/terraform-kubestack-testing/providers/Microsoft.ContainerService/managedClusters/kbstacctest-ops-westeurope]
Step #5 - "terraform apply": module.aks_zero.module.cluster.azurerm_kubernetes_cluster.current: Still modifying... [id=/subscriptions/803f877a-b0ff-49a7-afc7-...gedClusters/kbstacctest-ops-westeurope, 10s elapsed]
Step #5 - "terraform apply": module.aks_zero.module.cluster.azurerm_kubernetes_cluster.current: Still modifying... [id=/subscriptions/803f877a-b0ff-49a7-afc7-...gedClusters/kbstacctest-ops-westeurope, 20s elapsed]
Step #5 - "terraform apply": module.aks_zero.module.cluster.azurerm_kubernetes_cluster.current: Still modifying... [id=/subscriptions/803f877a-b0ff-49a7-afc7-...gedClusters/kbstacctest-ops-westeurope, 30s elapsed]
Step #5 - "terraform apply": module.aks_zero.module.cluster.azurerm_kubernetes_cluster.current: Still modifying... [id=/subscriptions/803f877a-b0ff-49a7-afc7-...gedClusters/kbstacctest-ops-westeurope, 40s elapsed]
Step #5 - "terraform apply": module.aks_zero.module.cluster.azurerm_kubernetes_cluster.current: Still modifying... [id=/subscriptions/803f877a-b0ff-49a7-afc7-...gedClusters/kbstacctest-ops-westeurope, 50s elapsed]
Step #5 - "terraform apply": module.aks_zero.module.cluster.azurerm_kubernetes_cluster.current: Still modifying... [id=/subscriptions/803f877a-b0ff-49a7-afc7-...gedClusters/kbstacctest-ops-westeurope, 1m0s elapsed]
Step #5 - "terraform apply": module.aks_zero.module.cluster.azurerm_kubernetes_cluster.current: Modifications complete after 1m5s [id=/subscriptions/803f877a-b0ff-49a7-afc7-eea135679b9a/resourcegroups/terraform-kubestack-testing/providers/Microsoft.ContainerService/managedClusters/kbstacctest-ops-westeurope]
Step #5 - "terraform apply": 
Step #5 - "terraform apply": Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

Expected Behavior

No diff after a successful apply.

Actual Behavior

Running apply will repeatedly remove the kube_dashboard block. Even though the block is optional according to the documentaion.

Steps to Reproduce

1. terraform apply

Running apply repeatedly will make the same change every time.

Important Factoids

No.

References

• #0000

[pst]

@tombuildsstuff I don't think question is the correct label here. This behavior has changed. Either between a provider release or due to a change on the Azure API side. Either way, the block is optional and should not cause a diff if not set.

Is this behavior possible related to this upstream change?

[tombuildsstuff]

hi @pst

Thanks for opening this issue.

Unfortunately the AKS API behaves differently for this field in different Azure Environments - where in Azure Public this is (at least, until Kubernetes 1.19, configurable) - but in other environments cannot be specified at all. As such unfortunately this field is intentionally not marked as computed, since Azure requires that this is configured.

Based on the document you've linked too, it sounds like Azure is trying to remove these differences by requiring cluster administrators to manage the deployment of the kubernetes dashboard - as such this field will be removed by AKS in the future (in Kubernetes 1.19 deployments), meaning that this field cannot be made Computed since being nil is a valid value.

Based on what I can tell, the current behaviour appears to be the "least bad" of the available options by having customers specify this in Azure Public for pre-1.19 clusters - and omitted for clusters in other Azure Environments. As such whilst I'd like to thank you for opening this issue, since this behaviour is due to the design of the AKS API I'm going to close this issue for the moment - but it should be possible to remove this diff by specifying the kube_dashboard add-on within the Terraform Configuration.

Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!