azurerm_kubernetes_cluster - Support for KMS arguments

[mkilchhofer]

Resolves #17957

Our security guys at @swisspost asked to implement Add Key Management Service (KMS) etcd encryption to an Azure Kubernetes Service (AKS) cluster. Unfortunately the provider did not implement it, so I implemented it for us and the community :-).

Example of implementation:

resource "azurerm_kubernetes_cluster" "aks" {
   
  # ...

  identity {
    type         = "UserAssigned"
    identity_ids = ["/subscriptions/11111111-1111-1111-1111-111111111111/resourceGroups/rg-my-resource-group/providers/Microsoft.ManagedIdentity/userAssignedIdentities/id-acctestaks"]
  }

  key_management_service {
    key_vault_key_id  = "https://kv-acctestaks.vault.azure.net/keys/acctestaks/dummykeyversion"
  }
}

[lonegunmanb]

Would raw["key_vault_resource_id"] here be nil?

[lonegunmanb]

Hi @mkilchhofer , since the key_vault_resource_id is an optional argument, would raw["key_vault_resource_id"] return a nil and cause panic here?

[mkilchhofer]

I didn't set it in my tests and did not get a panic here :-)

Will try to refactor this and remove it. (see comment of @stephybun #19893 (comment))

[mkilchhofer]

As I don't know if its possible to attach a debugger to a provider I tested it with adding some logging:

$ git di
diff --git a/internal/services/containers/kubernetes_cluster_resource.go b/internal/services/containers/kubernetes_cluster_resource.go
index 0f4a4d70d9..1a3959cff5 100644
--- a/internal/services/containers/kubernetes_cluster_resource.go
+++ b/internal/services/containers/kubernetes_cluster_resource.go
@@ -3,6 +3,7 @@ package containers
 import (
        "context"
        "encoding/base64"
+       "encoding/json"
        "fmt"
        "log"
        "strconv"
@@ -3427,6 +3428,18 @@ func expandKubernetesClusterAzureKeyVaultKms(d *pluginsdk.ResourceData, input []
        }

        raw := input[0].(map[string]interface{})
+       rawJson, _ := json.Marshal(raw)
+
+       log.Printf("[DEBUG] AzureKeyVaultKms - raw %v", raw)
+       log.Printf("[DEBUG] AzureKeyVaultKms - raw %v", string(rawJson))
+
+       log.Printf("[DEBUG] AzureKeyVaultKms - raw[key_vault_key_id]: %v", raw["key_vault_key_id"])
+       log.Printf("[DEBUG] AzureKeyVaultKms - raw[key_vault_key_id](string): %v", raw["key_vault_key_id"].(string))
+
+       log.Printf("[DEBUG] AzureKeyVaultKms - raw[key_vault_resource_id]: %v", raw["key_vault_resource_id"])
+       log.Printf("[DEBUG] AzureKeyVaultKms - raw[key_vault_resource_id](string): %v", raw["key_vault_resource_id"].(string))
+
        kvAccess := managedclusters.KeyVaultNetworkAccessTypes(raw["key_vault_network_access"].(string))

        azureKeyVaultKms := &managedclusters.AzureKeyVaultKms{

Output of this is:

$ grep AzureKeyVaultKms runlog3.log
2023-01-11T15:59:32.812+0100 [DEBUG] provider.terraform-provider-azurerm: AzureKeyVaultKms - raw map[key_vault_key_id:https://kv-aks-ci-m98ln0012-gj9j.vault.azure.net/keys/aks-ci-m98ln0012-gj9jjw-etcd-encryption/c059c6be19c441df9520ced5f08efc72 key_vault_network_access:Public key_vault_resource_id:]: timestamp=2023-01-11T15:59:32.811+0100
2023-01-11T15:59:32.812+0100 [DEBUG] provider.terraform-provider-azurerm: AzureKeyVaultKms - raw {"key_vault_key_id":"https://kv-aks-ci-m98ln0012-gj9j.vault.azure.net/keys/aks-ci-m98ln0012-gj9jjw-etcd-encryption/c059c6be19c441df9520ced5f08efc72","key_vault_network_access":"Public","key_vault_resource_id":""}: timestamp=2023-01-11T15:59:32.811+0100

2023-01-11T15:59:32.812+0100 [DEBUG] provider.terraform-provider-azurerm: AzureKeyVaultKms - raw[key_vault_key_id]: https://kv-aks-ci-m98ln0012-gj9j.vault.azure.net/keys/aks-ci-m98ln0012-gj9jjw-etcd-encryption/c059c6be19c441df9520ced5f08efc72: timestamp=2023-01-11T15:59:32.812+0100
2023-01-11T15:59:32.812+0100 [DEBUG] provider.terraform-provider-azurerm: AzureKeyVaultKms - raw[key_vault_key_id](string): https://kv-aks-ci-m98ln0012-gj9j.vault.azure.net/keys/aks-ci-m98ln0012-gj9jjw-etcd-encryption/c059c6be19c441df9520ced5f08efc72: timestamp=2023-01-11T15:59:32.812+0100

2023-01-11T15:59:32.812+0100 [DEBUG] provider.terraform-provider-azurerm: AzureKeyVaultKms - raw[key_vault_resource_id]:: timestamp=2023-01-11T15:59:32.812+0100
2023-01-11T15:59:32.812+0100 [DEBUG] provider.terraform-provider-azurerm: AzureKeyVaultKms - raw[key_vault_resource_id](string):: timestamp=2023-01-11T15:59:32.812+0100

As you can see in the first 2 lines, raw["key_vault_resource_id"] is empty string when unset in HCL.

[lonegunmanb]

If azureKeyVaultKms.KeyVaultResourceId is nil, then the dereference here would cause a panic.
If not, then what we're checking here is *azureKeyVaultKms.KeyVaultResourceId == ""

[stephybun]

Thanks for this PR @mkilchhofer. This is off to a great start, some changes need to be made to the schema but once that's done we can take another look through this 👍

[stephybun]

Thanks so much for making those changes @mkilchhofer!

I had one more comment regarding some nil checks in the flatten function which I unfortunately missed in the initial review, but once that's done this should be good to go!

No need to squash the commits, that will happen when this gets merged.

Great work and thanks again! 🙂

[stephybun]

Thanks for this @mkilchhofer LGTM 💯

[mkilchhofer]

Thank you for the approve and the merge. Your review was excellent 👍

[github-actions]

This functionality has been released in v3.40.0 of the Terraform Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.