DistributionAlreadyExists while creating multiple CloudFront distributions

[superuser5]

Terraform CLI and Terraform AWS Provider Version

./terraform -v
Terraform v1.2.3
on darwin_amd64

• provider registry.terraform.io/hashicorp/aws v4.19.0
• provider registry.terraform.io/hashicorp/http v2.2.0
• provider registry.terraform.io/hashicorp/null v3.1.1
• provider registry.terraform.io/hashicorp/random v3.3.1

Affected Resource(s)

• aws_cloudfront_distribution

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

# Copy-paste your Terraform configurations here - for large Terraform configs,
# please use a service like Dropbox and share a link to the ZIP file. For
# security, you can also encrypt the files using our GPG public key: https://keybase.io/hashicorp

resource "aws_cloudfront_distribution" "server" {
  count = 5
  wait_for_deployment = false # does not wait to be deployed and continue execution
  retain_on_delete = true

  is_ipv6_enabled     = true
  enabled = true

  logging_config {
    include_cookies = true
    bucket          = "${aws_s3_bucket.log_bucket.id}.s3.amazonaws.com"
    prefix          = "cloudfront_logs"
  }

## ref: https://www.terraform.io/docs/providers/aws/r/cloudfront_distribution.html
  origin {
    domain_name = aws_instance.server.public_dns 
    origin_id   = "server-${aws_instance.server.public_dns}" 

    custom_origin_config {
      http_port              = 80
      https_port             = 443
      origin_protocol_policy = "http-only"
      origin_ssl_protocols   = ["TLSv1.1"]
    }
  }

  default_cache_behavior {
    allowed_methods  = ["DELETE", "GET", "HEAD", "OPTIONS", "PATCH", "POST", "PUT"]
    cached_methods   = ["GET", "HEAD"]
    target_origin_id = "server-${aws_instance.server.public_dns}"

    forwarded_values {
      headers = ["*"]

      cookies {
        forward = "all"
      }

      query_string = true
    }

    # viewer_protocol_policy = "allow-all"
    viewer_protocol_policy = "redirect-to-https"
    min_ttl                = 0
    max_ttl                = 0
    default_ttl            = 0
    compress               = false
    smooth_streaming       = false
  }

  restrictions {
    geo_restriction {
      restriction_type = "none"
    }
  }

  price_class = "PriceClass_All"

  viewer_certificate {
    cloudfront_default_certificate = true
  }

  lifecycle {
    create_before_destroy = true
  }
}

Debug Output

Panic Output

│ Error: error creating CloudFront Distribution: DistributionAlreadyExists: The caller reference that you are using to create a distribution is associated with another distribution. Already exists: E337CUMHPXR6RL
│ status code: 409, request id: df488038-e219-4ed5-bb68-155a02fd7c02
│

Expected Behavior

Multiple distributions are created.

Actual Behavior

╷
│ Error: error creating CloudFront Distribution: DistributionAlreadyExists: The caller reference that you are using to create a distribution is associated with another distribution. Already exists: E337CUMHPXR6RL
│ status code: 409, request id: df488038-e219-4ed5-bb68-155a02fd7c02
│
│ with aws_cloudfront_distribution.server[3],
│ on server_cf.tf line 18, in resource "aws_cloudfront_distribution" "server":
│ 18: resource "aws_cloudfront_distribution" "server" {

Steps to Reproduce

1. terraform apply

Important Factoids

have custom setup settings

wait_for_deployment = false

count = var.number

retain_on_delete = false

References

• #0000

[trevorrea]

What's the use case behind this? I'm struggling to think of a reason for creating 5 identical CloudFront Distributions? As far as I know you can't have 2 distributions with the same domain_name but I could be wrong.

[justinretzolk]

Hey @superuser5 👋 To confirm what @trevorrea mentioned above, the domain_name must be unique, and is likely what is causing the issue here. If you're looking to create multiple distributions using a single configuration like this, you might consider using for_each rather than count, using a set of domain names as the value for for_each and then passing each.key for the domain_name argument. I believe that will get you around these errors.

[bryan-rhm]

as @justinretzolk said you can't create multiple distributions with same domain name according to the docs:

When you try to add an alternate domain name to a distribution but the alternate domain name is already in use on a different distribution, you get a CNAMEAlreadyExists error (One or more of the CNAMEs you provided are already associated with a different resource). For example, you get this error when you attempt to add www.example.com to a distribution, but www.example.com is already associated with a different distribution.

References:

Moving an alternate domain name to a different distribution

[lmf-mx]

Stumbled on this while looking into how uniqueness is handled between this module and Cloudfront. From testing, creating multiple identical distributions natively in AWS (i.e. console) works just fine when you are not using alternate domain names. The config above does not have aliases set.

Cloudfront generates unique domains names itself for each distribution. The ID is also unique.

From https://docs.aws.amazon.com/cli/latest/reference/cloudfront/create-distribution.html

CallerReference -> (string)

A unique value (for example, a date-time stamp) that ensures that the request can't be replayed.

If the value of CallerReference is new (regardless of the content of the DistributionConfig object), CloudFront creates a new distribution.

If CallerReference is a value that you already sent in a previous request to create a distribution, CloudFront returns a DistributionAlreadyExists error.

The error from @superuser5 is compaining about the caller reference not Service: AmazonCloudFront; Status Code: 409; Error Code: CNAMEAlreadyExists;.
The above comments about alternate domain names were confusing being new to Cloudfront configuration. This post is to clear up what is actually applicable with the above config for anyone who also lands here and not to justify the use case. That would require a way to determine if multiple objects should be created and then map those in some fashion to be able to maintain them over time.

[phene]

I just ran into this and it is not because CNAMEAlreadyExists. The TF resource did in fact create the CloudFront distribution, but got confused along the way and lost track of it. My best guess is that it's triggered during rate-limiting.