S3 Lifecycle Rule Unnecessarily Updates on Each Terraform Apply

[hashibot]

This issue was originally opened by @josh-padnick as hashicorp/terraform#9119. It was migrated here as part of the provider split. The original body of the issue is below.

---

I encountered an issue where adding a lifecycle configuration to an S3 Bucket causes terraform to keep destroying and re-adding the same lifecycle on every subsequent terraform apply.

Terraform Version

Terraform v0.7.4

Affected Resource(s)

• aws_s3_bucket

Terraform Configuration Files

resource "aws_s3_bucket" "cloudtrail" {
  bucket = "gruntwork-josh-test-1"
  force_destroy = true

  versioning {
    enabled = true
  }

  # Automatically delete a log file after X days.
  lifecycle_rule {
    id = "auto-delete-after-x-days"
    prefix = ""
    enabled = true
    abort_incomplete_multipart_upload_days = 5

    expiration {
      days = 5
      expired_object_delete_marker = true
    }

    noncurrent_version_expiration {
      days = 5
    }
  }
}

Debug Output

https://gist.github.com/josh-padnick/aea607bd08cc771344e5c9063f30a495

Panic Output

N/A

Expected Behavior

After the initial apply, subsequent terraform apply or terraform plan should result in no changes.

Actual Behavior

The S3 bucket is created successfully and without error on the initial terraform apply. But without any changes to the template, running terraform plan immediately after shows this pointless change:

~ aws_s3_bucket.cloudtrail
    lifecycle_rule.0.expiration.2217964487.date:                         "" => ""
    lifecycle_rule.0.expiration.2217964487.days:                         "" => "5"
    lifecycle_rule.0.expiration.2217964487.expired_object_delete_marker: "" => "true"
    lifecycle_rule.0.expiration.4129135488.date:                         "" => ""
    lifecycle_rule.0.expiration.4129135488.days:                         "5" => "0"
    lifecycle_rule.0.expiration.4129135488.expired_object_delete_marker: "false" => "false"

Steps to Reproduce

1. terraform apply
2. terraform plan

Important Factoids

Interestingly, I also discovered what is probably a race condition while preparing this issue. In about 25% of cases, doing a simple terraform apply would error out with the following:

1 error(s) occurred:

* aws_s3_bucket.cloudtrail: Error putting S3 lifecycle: NoSuchBucket: The specified bucket does not exist
    status code: 404, request id: 7B9E0FD8D09EAFDC

This is unrelated to this issue, but probably warrants a separate issue if one doesn't already exist.

[webdev]

Is this issue still open?

[wjam]

This is still an issue. The cause of the issue is that AWS doesn't support expired_object_delete_marker when expiration has already been configured.
See hashicorp/terraform#9119 (comment) for more details

[skynardo]

I ran into this same issue when trying to set expired_object_delete_marker = true in the same lifecycle rule that I was setting current version expiration and noncurrent_version_expiration. Terraform would run without errors but the plan would show setting the expired_object_delete_marker = false (see below)

expiration {
expired_object_delete_marker = true
}

lifecycle_rule.1.expiration.3591068768.expired_object_delete_marker: "true" => "false"

The solution is to create a second lifecycle rule that sets the expired_object_delete_marker = true

lifecycle_rule {
  id      = "ExpireDeleteMarkers"
  prefix  = ""
  enabled = true

  expiration {
    expired_object_delete_marker = true
  }
}

[github-actions]

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed. Maintainers can also remove the stale label.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!