Terraform shouldn't remove deposed resources that have the same id of a resource just created

[mildred]

Terraform Version

terraform 0.11.10

Terraform Configuration Files

resource "aws_launch_configuration" "etcd" {
  image_id      = "..."
  instance_type = "...}"
  key_name      = "..."

  security_groups = [...]

  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_autoscaling_group" "etcd" {
  name     = "etcd_server - ${aws_launch_configuration.etcd.name}"
  max_size = 1
  min_size = 1

  health_check_grace_period = 30
  health_check_type         = "EC2"
  desired_capacity          = 1

  force_delete    = true
  placement_group = "..."

  launch_configuration = "${aws_launch_configuration.etcd.id}"
  vpc_zone_identifier  = ["..."]

  lifecycle {
    create_before_destroy = true
  }
}

Actual Behavior

In the following scenario, terraform is removing the auto scaling group it just created:

• terraform starts and tries to create the ASG

• terraform creates the launch configuration "terraform-20190215140012211400000002"

• terraform creates an ASG with the name "etcd_server - terraform-20190215140012211400000002"

• the ASG health check fails, and terraform fails with an error like:

  aws_autoscaling_group.etcd: "etcd_server - terraform-20190215140012211400000002": Waiting up to 10m0s: Need at least 1 healthy instances in ELB, have 0. Most recent activity: ...

• the error is on the infrastructure, some changes are made to the infrastructure outside of terraform

• the ASG is deleted manually

• no modifications are made in terraform itself, terraform is executed again, it should work

• terraform sees that the ASG "etcd_server - terraform-20190215140012211400000002" does not exists (the previous one failed too) and it creates it

• terraform still remembers of the previous ASG named exactly the same and removes it

Log messages for this:

aws_autoscaling_group.etcd_server: Creating...
  arn:                                "" => "<computed>"
  default_cooldown:                   "" => "<computed>"
  desired_capacity:                   "" => "3"
  force_delete:                       "" => "true"
  health_check_grace_period:          "" => "60"
  health_check_type:                  "" => "ELB"
  launch_configuration:               "" => "terraform-20190215140012211400000002"
  load_balancers.#:                   "" => "1"
  load_balancers.3840307022:          "" => "etcd-lb"
  max_size:                           "" => "3"
  metrics_granularity:                "" => "1Minute"
  min_size:                           "" => "3"
  name:                               "" => "etcd_server - terraform-20190215140012211400000002"
  placement_group:                    "" => "Platform Spread Placement Group"
  protect_from_scale_in:              "" => "false"
  service_linked_role_arn:            "" => "<computed>"
  tag.#:                              "" => "1"
  tag.2542448327.key:                 "" => "Name"
  tag.2542448327.propagate_at_launch: "" => "true"
  tag.2542448327.value:               "" => "etcd"
  target_group_arns.#:                "" => "<computed>"
  vpc_zone_identifier.#:              "" => "3"
  vpc_zone_identifier.1826629713:     "" => "..."
  vpc_zone_identifier.3036906803:     "" => "..."
  vpc_zone_identifier.483652940:      "" => "..."
  wait_for_capacity_timeout:          "" => "10m"
  wait_for_elb_capacity:              "" => "3"
aws_autoscaling_group.etcd_server: Still creating... (10s elapsed)
aws_autoscaling_group.etcd_server: Still creating... (20s elapsed)
aws_autoscaling_group.etcd_server: Still creating... (30s elapsed)
aws_autoscaling_group.etcd_server: Still creating... (40s elapsed)
aws_autoscaling_group.etcd_server: Still creating... (50s elapsed)
aws_autoscaling_group.etcd_server: Still creating... (1m0s elapsed)
aws_autoscaling_group.etcd_server: Still creating... (1m10s elapsed)
aws_autoscaling_group.etcd_server: Still creating... (1m20s elapsed)
aws_autoscaling_group.etcd_server: Still creating... (1m30s elapsed)
aws_autoscaling_group.etcd_server: Still creating... (1m40s elapsed)
aws_autoscaling_group.etcd_server: Creation complete after 1m40s (ID: etcd_server - terraform-20190215140012211400000002)
aws_autoscaling_group.etcd_server.deposed: Destroying... (ID: etcd_server - terraform-20190107153707032400000002)
aws_autoscaling_group.etcd_server.deposed: Destroying... (ID: etcd_server - terraform-20190215140012211400000002)
aws_autoscaling_group.etcd_server (deposed #1): Still destroying... (ID: etcd_server - terraform-20190107153707032400000002, 10s elapsed)
aws_autoscaling_group.etcd_server (deposed #0): Still destroying... (ID: etcd_server - terraform-20190215140012211400000002, 10s elapsed)
aws_autoscaling_group.etcd_server (deposed #1): Still destroying... (ID: etcd_server - terraform-20190107153707032400000002, 20s elapsed)
aws_autoscaling_group.etcd_server (deposed #0): Still destroying... (ID: etcd_server - terraform-20190215140012211400000002, 20s elapsed)
aws_autoscaling_group.etcd_server (deposed #0): Still destroying... (ID: etcd_server - terraform-20190215140012211400000002, 30s elapsed)
aws_autoscaling_group.etcd_server (deposed #1): Still destroying... (ID: etcd_server - terraform-20190107153707032400000002, 30s elapsed)
aws_autoscaling_group.etcd_server (deposed #0): Still destroying... (ID: etcd_server - terraform-20190215140012211400000002, 40s elapsed)
aws_autoscaling_group.etcd_server (deposed #1): Still destroying... (ID: etcd_server - terraform-20190107153707032400000002, 40s elapsed)
aws_autoscaling_group.etcd_server (deposed #1): Still destroying... (ID: etcd_server - terraform-20190107153707032400000002, 50s elapsed)
aws_autoscaling_group.etcd_server (deposed #0): Still destroying... (ID: etcd_server - terraform-20190215140012211400000002, 50s elapsed)
aws_autoscaling_group.etcd_server (deposed #0): Still destroying... (ID: etcd_server - terraform-20190215140012211400000002, 1m0s elapsed)
aws_autoscaling_group.etcd_server (deposed #1): Still destroying... (ID: etcd_server - terraform-20190107153707032400000002, 1m0s elapsed)
aws_autoscaling_group.etcd_server (deposed #0): Still destroying... (ID: etcd_server - terraform-20190215140012211400000002, 1m10s elapsed)
aws_autoscaling_group.etcd_server (deposed #1): Still destroying... (ID: etcd_server - terraform-20190107153707032400000002, 1m10s elapsed)
aws_autoscaling_group.etcd_server (deposed #0): Still destroying... (ID: etcd_server - terraform-20190215140012211400000002, 1m20s elapsed)
aws_autoscaling_group.etcd_server (deposed #1): Still destroying... (ID: etcd_server - terraform-20190107153707032400000002, 1m20s elapsed)
aws_autoscaling_group.etcd_server.deposed: Destruction complete after 1m28s
aws_autoscaling_group.etcd_server (deposed #0): Still destroying... (ID: etcd_server - terraform-20190215140012211400000002, 1m30s elapsed)
aws_autoscaling_group.etcd_server (deposed #0): Still destroying... (ID: etcd_server - terraform-20190215140012211400000002, 1m40s elapsed)
aws_autoscaling_group.etcd_server (deposed #0): Still destroying... (ID: etcd_server - terraform-20190215140012211400000002, 1m50s elapsed)
aws_autoscaling_group.etcd_server.deposed: Destruction complete after 1m58s
aws_launch_configuration.etcd_server.deposed: Destroying... (ID: terraform-20190107153707032400000002)
aws_launch_configuration.etcd_server.deposed: Destruction complete after 0s

Apply complete! Resources: 1 added, 0 changed, 3 destroyed.

Here, terraform creates a resource and destroys it right after

Expected Behavior

Either:

• terraform detects that the deposed resource did not exist, and does not attempt to remove it
• terraform detects that one of the deposed resources has the same id as the resource just created, and handles the case by erroring out or ignoring the deposed resource

[jbardin]

Hi @mildred,

Thanks for filing the issue!

While Terraform will not be able to determine what constitutes a unique identifier for a resource, I think it might be a good idea to try checking if a new instance is completely equal to its deposed instance and remove it from the state.

[mildred]

While Terraform will not be able to determine what constitutes a unique identifier for a resource

When the name for an auto scaling group changes, terraform knows it has to create a new resource. Can't that information be used to determine that if an identical named resource was created, it's no longer to be removed ?

Another way to fix this without having terraform knows about unique resource identifiers, would be to check during the plan phase if deposed resources are still there, and if deposed resources are detected as not existing, remove them from the state at that moment.

[jbardin]

When the name for an auto scaling group changes, terraform knows it has to create a new resource. Can't that information be used to determine that if an identical named resource was created, it's no longer to be removed ?

Terraform does not have any way of knowing that a specific attribute must be unique between resources. The provider may have that information, but the best it can do is return an error when you attempt to create the duplicate resource.

The reason this doesn't come up often, is that if you cannot create multiple instances of a resource from the same config, you cannot use create_before_destroy. This is why aws_autoscaling_group has name_prefix , which is used in a most of the samples, so that each instance receives a unique name allowing create_before_destroy to work.

[salvianreynaldi]

Hi,
I just had this issue where an ASG deployment failed[1] because of the requested instance type not being available. When terraform apply is executed again[2] and the instance is available, terraform will complain as the ASG with the same name was already created.
If I delete the ASG manually (e.g. using AWS Console) and re-execute terraform[3], it will recreate-then-destroy the ASG as the ASG with that name, even though the ASG's ARN is different, is in the deposed state.
Maybe it's because terraform doesn't store deposed resource's ARN in its state, and it delete the resource by name https://github.com/terraform-providers/terraform-provider-aws/blob/master/aws/resource_aws_autoscaling_group.go#L979

If I use name_prefix for the ASG and do [2], will it generate a new name? EDIT: it is, as name prefix uses some kind of timestamp, so the execution will result in new names