Cluster gets recreated(deleted then created) for the latest version v2.3.0

[Andrei-sys]

Describe the Bug

When trying to upgrade kafka module to current latest version, it has an unexpected behaviour. It deletes the current kafka cluster, and try to recreate it. Also, the Security Group gets deleted and recreated with same rules. The only difference is that it changes its name from "default" to "cbd":

module.test-kafka-cluster.module.msk_cluster.module.security_group.aws_security_group.cbd[0] will be created

• resource "aws_security_group" "cbd" {
  • arn = (known after apply)
  • description = "Managed by Terraform"
  • egress = (known after apply)
  • id = (known after apply)
  • ingress = (known after apply)
  • name = (known after apply)
  • name_prefix = (known after apply)
  • owner_id = (known after apply)
  • revoke_rules_on_delete = false
  • tags = {
    • "Environment" = "test-kafka-upgrade"
    • "ManagedBy" = "Terraform"
    • "Name" = "devops-test-kafka-upgrade"
    • "Namespace" = "devops"
    • "Team" = "devops"
      }
  • tags_all = {
    • "Environment" = "test-kafka-upgrade"
    • "ManagedBy" = "Terraform"
    • "Name" = "devops-test-kafka-upgrade"
    • "Namespace" = "devops"
    • "Team" = "devops"
      }

module.test-kafka-cluster.module.msk_cluster.module.security_group.random_id.rule_change_forces_new_security_group[0] will be created

• resource "random_id" "rule_change_forces_new_security_group" {
  • b64_std = (known after apply)
  • b64_url = (known after apply)
  • byte_length = 3
  • dec = (known after apply)
  • hex = (known after apply)
  • id = (known after apply)

module.test-kafka-cluster.module.msk_cluster.module.broker_security_group.aws_security_group.default[0] will be destroyed

(because aws_security_group.default is not in configuration)

• resource "aws_security_group" "default" {

  • arn = "arn:aws:ec2:AWS-AZ:AWS-ACCOUNT-ID:security-group/sg-SG-ID" -> null

  • description = "MSK broker access" -> null

    module.test-kafka-cluster.module.msk_cluster.aws_msk_cluster.default[0] must be replaced

-/+ resource "aws_msk_cluster" "default" {

          ~ broker_node_group_info {
      ~ ebs_volume_size = 20 -> (known after apply)
      ~ security_groups = [
          - "sg-0e1e8b46171217159",
        ] -> (known after apply) # forces replacement
        # (3 unchanged attributes hidden)
        # (2 unchanged blocks hidden)
    }

Expected Behavior

Upgrade should've been smooth, and the attempt of deleting and creating again the kafka cluster should never happen.

Steps to Reproduce

Tried to upgrade from kafka module from version v1.3.1 to v2.3.0.

Screenshots

No response

Environment

No response

Additional Context

No response

[dmitriy-lukyanchikov]

set additional variable and it will work as before

  security_group_create_before_destroy = false
  preserve_security_group_id = true

[Andrei-sys]

Still the same behaviour after added those vars...It tdeletes the cluster and the security group, and tries to create them again. This means downtime and maybe lost of data...right?

module.test-kafka-cluster.module.msk_cluster.aws_msk_cluster.default[0] must be replaced

-/+ resource "aws_msk_cluster" "default" {

module.test-kafka-cluster.module.msk_cluster.module.broker_security_group.aws_security_group.default[0] will be destroyed

(because aws_security_group.default is not in configuration)

• resource "aws_security_group" "default" {

[quercusilvam]

Some extra work needed. I figured it out.

You need to add moved block so terraform sees that there is rename of a security group module (which forces cluster recreation later).

moved {
  from = module.msk_cluster.module.broker_security_group
  to   = module.msk_cluster.module.security_group
}

Also you need the block that @dmitriy-lukyanchikov mentioned. In my case I need to set description of sg as well (as default one changed and that also forcing recreation)

module "msk_cluster" {
  source  = "cloudposse/msk-apache-kafka-cluster/aws"
  version = "2.3.0"

  security_group_create_before_destroy = false
  preserve_security_group_id           = true
  security_group_description           = "MSK broker access"
  ...
}

As a result only my aws_security_group_rules were recreated. Also new aws_msk_configuration was added as default name pattern for it changed (added kafka version). I can live with that.

Sorry guys but this should be mentioned in the 2.0.0 release notes. It's not easy to figure it out without some debugging.
I'm not sure but maybe you can put moved block (with some modification) inside the module code. But as users can set create_security_group to false so it should be well tested.

[Andrei-sys]

Hello, indeed, you're right. I did everything of the above and it avoided the deletion of the cluster and the SG. Thank you for that.
Still one question ongoing: Is there any possibility to avoid changing the name/description of the configuration?

module.test-kafka-cluster.module.msk_cluster.aws_msk_configuration.config[0] must be replaced

+/- resource "aws_msk_configuration" "config" {
~ description = "Manages an Amazon Managed Streaming for Kafka configuration" -> "Configuration for Amazon Managed Streaming for Kafka"
~ name = "devops-test-kafka-upgrade" -> "devops-test-kafka-upgrade-3-2-0" # forces replacement

[quercusilvam]

I'm glad that my codehelped you :)

About this aws_msk_configuration - quick look shows that they added join with kafka_version variable in the code (which is root cause to create new config version). From main.tf:

name           = join("-", [module.this.id, replace(var.kafka_version, ".", "-")])

Because kafka_version cannot be null - IMHO it cannot be avoided.
You can fork the code and change this line to omit this join/replace - so you will keep your old name.