Prefix is mandatory in aws_s3_bucket_replication_configuration resource

[c-elliott]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Reproduced with two versions:
Terraform 1.0.11 with aws 3.67.0
Terraform 0.13.6 and aws 3.67.0

chris-mbp: us-west-1-rc> terraform -v
Terraform v0.13.6
+ provider registry.terraform.io/hashicorp/aws v3.67.0
+ provider registry.terraform.io/hashicorp/template v2.2.0

Affected Resource(s)

• aws_s3_bucket_replication_configuration

Terraform Configuration Files

resource "aws_s3_bucket_replication_configuration" "bucket-replication" {
  role   = data.aws_iam_role.bucket-replication-role.arn
  bucket = data.aws_s3_bucket.source-bucket.id
  rule {
    id     = "bucket-replication"
    prefix = ""
    status = "Enabled"
    destination {
      bucket = aws_s3_bucket.replicated-bucket.arn
    }
  }
}

Expected Behavior

A replication rule should be created with a scope for the entire bucket when "prefix" is not specified or is set to an empty string like in the example above. This is how replication rules behave when creating them within an aws_s3_bucket resource.

Actual Behavior

Terraform apply fails with Invalid XML error:

Error: error creating S3 replication configuration for bucket (source-bucket): MalformedXML: The XML you provided was not well-formed or did not validate against our published schema
	status code: 400, request id: G0V996BJCB67WFHY, host id: HKx7zLDfemY9+TgK09fiM36NakBE1RjU18cHuMMkAGlNaFQvRpXfPWuAv6ws0vf+6rXexX+6pvI=

The only way to avoid this error is to specify something for "prefix", which isn't useful when I want to replicate everything in the bucket.

Additional example

This is the result when I create a replication rule with a prefix of "foo" using terraform, modify it in the console to have no prefix and run "terraform apply". It does not see prefix at all, so it should also accept configuration with no prefix when applying.

      - rule {
          - id       = "source-bucket" -> null
          - priority = 0 -> null
          - status   = "Enabled" -> null

          - destination {
              - bucket = "arn:aws:s3:::source-bucket" -> null
            }
        }
      + rule {
          + id     = "replicated-bucket"
          + prefix = "foo"
          + status = "Enabled"

          + destination {
              + bucket = "arn:aws:s3:::replicated-bucket"
            }
        }
    }

Steps to Reproduce

1. terraform apply

References

The documentation states prefix should be optional:

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_replication_configuration#prefix

[cablespaghetti]

This looks very similar to this PR from 2018 (for the aws_s3_bucket block) #6344

[rymancl]

I'm still running into this as of v3.71.0.

╷
│ Error: error creating S3 replication configuration for bucket (my-primary-bucket): MalformedXML: The XML you provided was not well-formed or did not validate against our published schema
│ status code: 400, request id: , host id:
│
│ with aws_s3_bucket_replication_configuration.primary_to_replica,
│ on s3-primary.tf line 53, in resource "aws_s3_bucket_replication_configuration" "primary_to_replica":
│ 53: resource "aws_s3_bucket_replication_configuration" "primary_to_replica" {
│
╵

-> terraform version
Terraform v1.1.1
on linux_amd64
+ provider registry.terraform.io/hashicorp/aws v3.71.0

Terraform configuration (primary to replica)

resource "aws_s3_bucket_replication_configuration" "primary_to_replica" {
  role   = aws_iam_role.replication_primary.arn
  bucket = aws_s3_bucket.terraform_state.id

  rule {
    id     = "Primary_To_Replica"
    status = "Enabled"
    
    filter {}

    delete_marker_replication {
      status = "Enabled"
    }

    existing_object_replication {
      status = "Enabled"
    }

    destination {
      bucket        = aws_s3_bucket.terraform_state_replica.arn
      storage_class = "STANDARD"
    }
  }
}

Terraform plan (primary to replica)

  + resource "aws_s3_bucket_replication_configuration" "primary_to_replica" {
      + bucket = "my-primary-bucket"
      + id     = (known after apply)
      + role   = "arn:aws:iam::ACCOUNTID:role/primary-replication-role"

      + rule {
          + id     = "Primary_To_Replica"
          + status = "Enabled"

          + delete_marker_replication {
              + status = "Enabled"
            }

          + destination {
              + bucket        = "arn:aws:s3:::my-replica-bucket"
              + storage_class = "STANDARD"
            }

          + existing_object_replication {
              + status = "Enabled"
            }

          + filter {
            }
        }
    }

FWIW, the replica to primary configuration in the same module worked. The only difference is no existing_object_replication here.

Terraform configuration (replica to primary)

resource "aws_s3_bucket_replication_configuration" "replica_to_primary" {
  provider = aws.replica

  role   = aws_iam_role.replication_replica.arn
  bucket = aws_s3_bucket.terraform_state_replica.id

  rule {
    id     = "Replica_To_Primary"
    status = "Enabled"
    
    filter {}

    delete_marker_replication {
      status = "Enabled"
    }

    destination {
      bucket        = aws_s3_bucket.terraform_state.arn
      storage_class = "STANDARD"
    }
  }
}

Terraform plan (replica to primary)

+ resource "aws_s3_bucket_replication_configuration" "replica_to_primary" {
      + bucket = "my-replica-bucket"
      + id     = (known after apply)
      + role   = "arn:aws:iam::ACCOUNTID:role/replica-replication-role"

      + rule {
          + id     = "Replica_To_Primary"
          + status = "Enabled"

          + delete_marker_replication {
              + status = "Enabled"
            }

          + destination {
              + bucket        = "arn:aws:s3:::my-primary-bucket"
              + storage_class = "STANDARD"
            }

          + filter {
            }
        }
    }

EDIT: Confirmed removing existing_object_replication from primary allowed the apply to succeed. I suspect this is not enabled for our account. I'm going to contact support to check.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.