RDS encrypted Snapshot restore uses snapshot's kms key

[ghost]

This issue was originally opened by @nomeelnoj as hashicorp/terraform#18984. It was migrated here as a result of the provider split. The original body of the issue is below.

---

Terraform Version

Terraform v0.11.8

Terraform Configuration Files

resource "aws_kms_key" "kms" {
  description  = "rds-tf-encrypt/${var.env_prefix}"
}

resource "random_id" "db_snapshot_suffix" {
  keepers = {
    rds_snapshot = "${var.snapshot_identifier}"
  }

  byte_length = 8
}

# Create RDS instance
resource "aws_db_instance" "rds" {
  identifier                = "${var.name_prefix}-${var.env_prefix}${var.version_prefix}"
  allocated_storage         = "${var.storage}"
  engine                    = "${var.engine}"
  engine_version            = "${lookup(var.engine_version, var.engine)}"
  instance_class            = "${var.instance_class}"
  publicly_accessible       = "${var.publicly_accessible}"
  backup_retention_period   = "${var.backup_retention_period}"
  apply_immediately         = "${var.apply_immediately}"
  multi_az                  = "${var.multi_az}"
  storage_type              = "${var.storage_type}"
  storage_encrypted         = true
  kms_key_id                = "${aws_kms_key.kms.arn}"
  final_snapshot_identifier = "${var.final_snapshot_identifier}-${var.env_prefix}-${random_id.db_snapshot_suffix.hex}"
  skip_final_snapshot       = "${var.env_prefix == "prd" ? false : true}"
  name                      = "${var.db_name}"
  username                  = "${var.username}"
  password                  = "${var.password}"
  vpc_security_group_ids    = ["${var.vpc_security_group_ids}"]
  db_subnet_group_name      = "${aws_db_subnet_group.default.name}"
  parameter_group_name      = "${var.parameter_group_name}"
  monitoring_interval       = "${var.monitoring_interval}"
  monitoring_role_arn       = "${var.monitoring_role_arn}"
  snapshot_identifier       = "${var.snapshot_identifier}"

Expected Behavior

When running the TF above, the new database should be encrypted with the KMS key provided, not the key from the snapshot.

Actual Behavior

The new RDS instance is created using the snapshot's KMS key for encryption

Steps to Reproduce

Additional Context

To update an RDS encryption key, you can create a copy of the snapshot and change the key for the copy. Couldn't you update TF to first copy the snapshot and apply the new key to the snapshot and then restore from the snapshot if both kms_key_id and snapshot_identifier are provided?

[bflad]

Hi @nomeelnoj 👋 Sorry you ran into trouble here. Your timing is actually pretty good though as #6012 was recently merged to fix this exact issue and it was just released in version 1.39.0 of the AWS provider this afternoon. 👍

[nomeelnoj]

Looks like this was integrated for aws_db_cluster, but not aws_db_instance. Is the functionality for db_instance expected soon as well?

[bflad]

Ah, good catch, @nomeelnoj 😅 you are correct. Sorry about that - reopening. The API does not support KmsKeyId directly with RestoreDBInstanceFromDBSnapshot so this is indeed more complicated than working with Aurora clusters. Regardless of what we do here, I would suggest opening a support case with AWS to see if they might implement that into the API directly.

As for the current situation, I'm not sure we would accept having the aws_db_instance resource create an "unmanaged" or temporary snapshot with the specific configuration. I think we would prefer a new aws_db_snapshot_copy or similar resource, that managed the lifecycle of a snapshot copy separately. That new resource would also allow for copying between regions, which is nice.

[nomeelnoj]

Could it be configured to automatically create a copy of the snapshot, assign the new KMS key, then restore from that copy, then delete that copy? Or is that too much going on behind the scenes?

[bflad]

Hi again 👋 Sorry for the delayed response.

Could it be configured to automatically create a copy of the snapshot, assign the new KMS key, then restore from that copy, then delete that copy? Or is that too much going on behind the scenes?

In general, yes, we strongly prefer that Terraform resources only manage one piece of infrastructure (even temporarily). Operators have expectations that no other infrastructure is being created outside of what is declared in their Terraform configuration. If any part of the temporary processing failed, Terraform would have no method for tracking or handling the temporary infrastructure left behind.

The good news here is that it looks like a community member has contributed what we would expect the solution to look like here in a new aws_db_snapshot_copy resource: #9885 and an initial pull request #9886

Since that issue and pull request follow our recommended path forward here, I'm going to opt to close this issue so we can consolidate discussions and efforts in those. Thanks again for submitting this.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!