google_notebooks_instance seems to always force recreation when CMEK key encryption is set.

[Karrot96]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

Version v0.15.3

Affected Resource(s)

• google_notebooks_instance

Terraform Configuration Files

module "notebook_kms" {
  source = "<path_to_our_hsm_kms_key_file>"

  location              = "europe_west2"
  project_id            = var.project_id
  keyring               = "notebook"
  key                   = "key"
  owners                = ["group:${var.admin_group_email}"]
  encrypters_decrypters = ["serviceAccount:service-${data.google_project.current.number}@compute-system.iam.gserviceaccount.com"]
}


resource "google_service_account" "notebooks" {
  account_id = "notebooks"
}

resource "google_notebooks_instance" "notebook_instance" {
  name         = "notebook"
  location     = "europe-west2-a"
  machine_type = "e2-highmem-16"
  container_image {
    repository = "gcr.io/deeplearning-platform-release/base-cpu"
    tag        = "latest"
  }

  service_account = google_service_account.notebooks.email

  no_public_ip    = true
  no_proxy_access = false

  network = local.network
  subnet  = local.subnet

  boot_disk_type    = "PD_SSD"
  boot_disk_size_gb = 200

  disk_encryption = "CMEK"
  kms_key         = module.notebook_kms.key_id

}

Our hsm_kms_key module

variable "keyring" {
}

variable "key" {
}

variable "location" {
}

variable "project_id" {
}

variable "owners" {
  type = list(string)
}

variable "encrypters_decrypters" {
  type    = list(string)
  default = []
}

variable "key_rotation_period" {
  default = "2592000s" # Once every 30 days
}

module "kms" {
  source  = "terraform-google-modules/kms/google"
  version = "~> 1.2"

  project_id = var.project_id
  location   = var.location
  keyring    = var.keyring
  keys       = [var.key]

  prevent_destroy      = true
  key_protection_level = "HSM"
  key_rotation_period  = var.key_rotation_period

  set_owners_for = [var.key]
  owners         = [join(",", var.owners)]

  set_decrypters_for = [var.key]
  decrypters         = [join(",", var.encrypters_decrypters)]

  set_encrypters_for = [var.key]
  encrypters         = [join(",", var.encrypters_decrypters)]
}

output "key_id" {
  value = module.kms.keys[var.key]

  # Make sure all permissions are setup prior to the key being used
  depends_on = [module.kms]
}

Debug Output

https://gist.github.com/Karrot96/4f817f93699140699e45233fba8673bc

Expected Behavior

Having previously run terraform apply it created the VM with the CMEK key. This is confirmed by checking in the google VM dashboad. Given there is nothing causing a change here we would expect to terraform apply return no changes. Instead it is suggesting a change is required.

Actual Behavior

When ever you run a terraform apply or a terraform plan it suggests that an addition of the CMEK key is required.

Steps to Reproduce

1.Create a google_notebooks_instance with disk_encryption set to CMEK.

1. run terraform apply
2. rerun terraform apply

Important Factoids

This is happening when we are authenticating as a user. We have managed to get round this temporarily by using the ignore_changes option. This is quite a painful issue if you accidentally run this as it deletes the data from the disk if you happen to have anything saved on there.

b/303399836

[edwardmedia]

Both disk_encryption and kms_key are missing in the API response. Added label upstream

[pandanos]

We have the same error! Please fix :)

[gxa138]

This issue has not been fixed. I have tested with the google provider v4.15.0

[melinath]

This should be resolved by GoogleCloudPlatform/magic-modules#9915

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.