Replacing instance doesn't detach network interface

[andrewmiskell]

Description

Please provide a clear and concise description of the issue you are encountering, and a reproduction of your configuration (see the examples/* directory for references that you can copy+paste and tailor to match your configs if you are unable to copy your exact configuration). The reproduction MUST be executable by running terraform init && terraform apply without any further changes.

If your request is for a new feature, please use the Feature request template.

• ✋ I have searched the open/closed issues and my issue is not listed.

⚠️ Note

Before you submit an issue, please perform the following first:

1. Remove the local .terraform directory (! ONLY if state is stored remotely, which hopefully you are following that best practice!): rm -rf .terraform/
2. Re-initialize the project root to pull down modules: terraform init
3. Re-attempt your terraform plan or apply and check if the issue still persists

Versions

• Module version [Required]: 6.0.2

• Terraform version:
  Terraform v1.12.2

• Provider version(s):
  provider registry.terraform.io/hashicorp/aws v6.0.0
  provider registry.terraform.io/hashicorp/random v3.7.2

Reproduction Code [Required]

module "ec2" {
  ami                = "ami-016a4219ed8998a27"
  availability_zone  = random_shuffle.availability_zone.result[0]
  enable_volume_tags = false
  instance_type      = "t3.large"
  key_name           = "xxxx"
  monitoring         = true
  name               = "lm01-dev"
  network_interface = {
    0 = {
      network_interface_id  = aws_network_interface.this.id
      delete_on_termination = false
    }
  }
  root_block_device = {
    delete_on_termination = false
    encrypted             = true
    tags = {
      Name = "lm01-dev - root volume"
    }
  }
  source  = "terraform-aws-modules/ec2-instance/aws"
  version = "6.0.2"
}
resource "aws_network_interface" "this" {
  security_groups = [
    module.default_sg.security_group_id,
    module.license_manager_sg.security_group_id,
  ]
  subnet_id = var.vpc_private_subnets[0]
  tags = {
    Name = "lm01-dev - primary eni"
  }
}

Steps to reproduce the behavior:

Are you using workspaces? --> No
Have you cleared the local cache (see Notice section above)? --> Yes
List steps in order that led up to the issue you encountered --> Upgraded AWS provider to 6.0 and Module to 6.0.2

Expected behavior

Network interface should be detached from the EC2 instance before terminating the instance, or EC2 instance should be terminated before attempting to create the replacement instance.

This was the behavior seen in the 5.x AWS provider/5.x EC2 module

Actual behavior

Terraform attempts to create the new EC2 instance before the old EC2 instance is terminated, so the ENI attachment is still active on the old instance and cannot be attached to the new instance.

Terminal Output Screenshot(s)

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement
+/- create replacement and then destroy

Terraform will perform the following actions:

  # aws_volume_attachment.data must be replaced
-/+ resource "aws_volume_attachment" "data" {
      ~ id                             = "vai-2211139420" -> (known after apply)
      ~ instance_id                    = "i-012c083628e6f2e31" -> (known after apply) # forces replacement
        # (4 unchanged attributes hidden)
    }

  # module.alb.aws_lb_target_group_attachment.this["instance"] must be replaced
-/+ resource "aws_lb_target_group_attachment" "this" {
      ~ id               = "arn:aws:elasticloadbalancing:us-east-1:000000000000:targetgroup/lm20250702175047052200000004/3002cb8aba7ddc5c-20250702175047989900000005" -> (known after apply)
      ~ target_id        = "i-012c083628e6f2e31" -> (known after apply) # forces replacement
        # (3 unchanged attributes hidden)
    }

  # module.ec2.aws_instance.this[0] must be replaced
+/- resource "aws_instance" "this" {
      ~ ami                                  = "ami-016a4219ed8998a27" -> "ami-02a416d187b865630" # forces replacement
      ~ arn                                  = "arn:aws:ec2:us-east-1:000000000000:instance/i-012c083628e6f2e31" -> (known after apply)
      ~ associate_public_ip_address          = false -> (known after apply)
      ~ disable_api_stop                     = false -> (known after apply)
      ~ disable_api_termination              = false -> (known after apply)
      ~ ebs_optimized                        = false -> (known after apply)
      + enable_primary_ipv6                  = (known after apply)
      - hibernation                          = false -> null
      + host_id                              = (known after apply)
      + host_resource_group_arn              = (known after apply)
      + iam_instance_profile                 = (known after apply)
      ~ id                                   = "i-012c083628e6f2e31" -> (known after apply)
      ~ instance_initiated_shutdown_behavior = "stop" -> (known after apply)
      + instance_lifecycle                   = (known after apply)
      ~ instance_state                       = "running" -> (known after apply)
      ~ ipv6_address_count                   = 0 -> (known after apply)
      ~ ipv6_addresses                       = [] -> (known after apply)
      + outpost_arn                          = (known after apply)
      + password_data                        = (known after apply)
      + placement_group                      = (known after apply)
      ~ placement_partition_number           = 0 -> (known after apply)
      ~ primary_network_interface_id         = "eni-0a82880c9cebfc060" -> (known after apply)
      ~ private_dns                          = "ip-10-0-241-150.ec2.internal" -> (known after apply)
      ~ private_ip                           = "10.0.241.150" -> (known after apply)
      + public_dns                           = (known after apply)
      + public_ip                            = (known after apply)
      ~ secondary_private_ips                = [] -> (known after apply)
      ~ security_groups                      = [] -> (known after apply)
      - source_dest_check                    = true -> null
      + spot_instance_request_id             = (known after apply)
      ~ subnet_id                            = "subnet-03c67f331e99aced0" -> (known after apply)
        tags                                 = {
            "Name" = "lm01-dev"
        }
      ~ tenancy                              = "default" -> (known after apply)
      + user_data_base64                     = (known after apply)
      ~ vpc_security_group_ids               = [
          - "sg-05d4503cdee46b771",
          - "sg-0b2024d26ec7b320e",
        ] -> (known after apply)
        # (8 unchanged attributes hidden)

      ~ capacity_reservation_specification (known after apply)
      - capacity_reservation_specification {
          - capacity_reservation_preference = "open" -> null
        }

      ~ cpu_options (known after apply)
      - cpu_options {
          - core_count       = 1 -> null
          - threads_per_core = 2 -> null
            # (1 unchanged attribute hidden)
        }

      ~ credit_specification {
          - cpu_credits = "unlimited" -> null
        }

      ~ ebs_block_device (known after apply)
      - ebs_block_device {
          - delete_on_termination = false -> null
          - device_name           = "/dev/sdf" -> null
          - encrypted             = true -> null
          - iops                  = 3000 -> null
          - kms_key_id            = "arn:aws:kms:us-east-1:000000000000:key/8bf36ae0-6629-4d25-8f77-75171db486e3" -> null
          - tags                  = {
              - "Name"        = "lm01-dev - data volume"
              - "environment" = "dev"
              - "managed_by"  = "terraform"
              - "project"     = "license-management"
              - "stack_id"    = "5cf9d2d2-75be-4d70-a9c2-801717c88e38"
              - "stack_path"  = "/stacks/development/license_management/compute/license_manager"
              - "stack_repo"  = "terramate-infrastructure"
            } -> null
          - tags_all              = {
              - "Name"        = "lm01-dev - data volume"
              - "environment" = "dev"
              - "managed_by"  = "terraform"
              - "project"     = "license-management"
              - "stack_id"    = "5cf9d2d2-75be-4d70-a9c2-801717c88e38"
              - "stack_path"  = "/stacks/development/license_management/compute/license_manager"
              - "stack_repo"  = "terramate-infrastructure"
            } -> null
          - throughput            = 125 -> null
          - volume_id             = "vol-085e4bb774b2a9ba0" -> null
          - volume_size           = 100 -> null
          - volume_type           = "gp3" -> null
            # (1 unchanged attribute hidden)
        }

      ~ enclave_options {
          ~ enabled = false -> (known after apply)
        }

      ~ ephemeral_block_device (known after apply)

      ~ instance_market_options (known after apply)

      ~ maintenance_options (known after apply)
      - maintenance_options {
          - auto_recovery = "default" -> null
        }

      ~ metadata_options {
          ~ instance_metadata_tags      = "disabled" -> (known after apply)
            # (4 unchanged attributes hidden)
        }

      ~ private_dns_name_options (known after apply)
      - private_dns_name_options {
          - enable_resource_name_dns_a_record    = false -> null
          - enable_resource_name_dns_aaaa_record = false -> null
          - hostname_type                        = "ip-name" -> null
        }

      ~ root_block_device {
          ~ device_name           = "/dev/sda1" -> (known after apply)
          ~ iops                  = 3000 -> (known after apply)
          ~ kms_key_id            = "arn:aws:kms:us-east-1:000000000000:key/8bf36ae0-6629-4d25-8f77-75171db486e3" -> (known after apply)
            tags                  = {
                "Name" = "lm01-dev - root volume"
            }
          ~ tags_all              = {
              - "Name"        = "lm01-dev - root volume"
              - "environment" = "dev"
              - "managed_by"  = "terraform"
              - "project"     = "license-management"
              - "stack_id"    = "5cf9d2d2-75be-4d70-a9c2-801717c88e38"
              - "stack_path"  = "/stacks/development/license_management/compute/license_manager"
              - "stack_repo"  = "terramate-infrastructure"
            } -> (known after apply)
          ~ throughput            = 125 -> (known after apply)
          ~ volume_id             = "vol-01fc046dec8c327c7" -> (known after apply)
          ~ volume_size           = 32 -> (known after apply)
          ~ volume_type           = "gp3" -> (known after apply)
            # (2 unchanged attributes hidden)
        }

        # (2 unchanged blocks hidden)
    }

Plan: 3 to add, 0 to change, 3 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_volume_attachment.data: Destroying... [id=vai-2211139420]
module.alb.aws_lb_target_group_attachment.this["instance"]: Destroying... [id=arn:aws:elasticloadbalancing:us-east-1:000000000000:targetgroup/lm20250702175047052200000004/3002cb8aba7ddc5c-20250702175047989900000005]
module.alb.aws_lb_target_group_attachment.this["instance"]: Destruction complete after 1s
aws_volume_attachment.data: Still destroying... [id=vai-2211139420, 00m10s elapsed]
aws_volume_attachment.data: Still destroying... [id=vai-2211139420, 00m20s elapsed]
aws_volume_attachment.data: Still destroying... [id=vai-2211139420, 00m30s elapsed]
aws_volume_attachment.data: Still destroying... [id=vai-2211139420, 00m40s elapsed]
aws_volume_attachment.data: Still destroying... [id=vai-2211139420, 00m50s elapsed]
aws_volume_attachment.data: Destruction complete after 51s
module.ec2.aws_instance.this[0]: Creating...
╷
│ Error: creating EC2 Instance: operation error EC2: RunInstances, https response error StatusCode: 400, RequestID: efa8cb14-0b50-4d8c-a5e7-e9e6a88fa41a, api error InvalidNetworkInterface.InUse: Interface: [eni-0a82880c9cebfc060] in use.
│
│   with module.ec2.aws_instance.this[0],
│   on .terraform/modules/ec2/main.tf line 43, in resource "aws_instance" "this":
│   43: resource "aws_instance" "this" {
│
╵
Error: one or more commands failed
> execution failed: running /Users/andrewmiskell/.asdf/shims/terraform apply (in /stacks/development/license_management/compute/license_manager): exit status 1

[bryantbiggs]

this is not something that is controlled by the module - I would post an issue on the AWS provider or search for an existing/prior issue there

[andrewmiskell]

Thanks!

I've done some more digging and discovered it's actually the target_group_attachment in the alb module that's causing the problem. If I let the module handle creating the attachment, terraform wants to create the ec2 instance before destroying the old one. If I handle the attachment outside of the module, the ec2 instance is destroyed first and then recreated.

I'll file an issue on the alb repository and see if this is expected behavior as I'm not sure why it's acting differently when the attachment is handled outside of the module.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.