Breaking change for several resources from 2.51.0 → 2.52.0 #12332

timvisher · 2020-03-10T15:30:19Z

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Versions

terraform -v
Terraform v0.12.23
+ provider.aws v2.51.0
+ provider.kubernetes v1.11.1
+ provider.template v2.1.2

to

terraform -v
Terraform v0.12.23
+ provider.aws v2.52.0
+ provider.kubernetes v1.11.1
+ provider.template v2.1.2

Relevant bit: provider.aws v2.51.0 → 2.52.0.

Affected Resource(s)

aws_iam_policy_document
aws_iam_policy
aws_s3_bucket

Terraform Configuration Files

data "aws_iam_policy_document" "XXX" {
  statement {
    actions = ...
    resources = [
      ...
    ]
  }
}

resource "aws_iam_policy" "XXX" {
  name   = "XXX"
  path   = "/"
  policy = data.aws_iam_policy_document.XXX.json
}

resource "aws_s3_bucket" "XXX" {
  bucket = ...
  acl    = "private"
  region = "us-east-1"

  tags = {
    Name = ...
  }
}

Expected Behavior

No changes. Infrastructure is up-to-date.

Actual Behavior

------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place
 <= read (data resources)

Terraform will perform the following actions:

  # module.XXX.data.aws_iam_policy_document.XXX will be read during apply
  # (config refers to values not yet known)
 <= data "aws_iam_policy_document" "deployment"  {
      + id   = (known after apply)
      + json = (known after apply)

      + statement {
          + actions   = [
              + ...
            ]
          + resources = [
              + ...
            ]
        }
    }

  # module.services.aws_iam_policy.deployment will be updated in-place
  ~ resource "aws_iam_policy" "XXX" {
        ...
      ~ policy      = jsonencode(
            {
              ...
            }
        ) -> (known after apply)
    }

  # module.XXX.aws_s3_bucket.XXX will be updated in-place
  ~ resource "aws_s3_bucket" "XXX" {
        ...

      - grant {
          ...
        }

        ...
    }

Plan: 0 to add, 7 to change, 0 to destroy.

------------------------------------------------------------------------

Steps to Reproduce

Bump AWS Provider from 2.51.0 to 2.52.0 and run a plan.

The text was updated successfully, but these errors were encountered:

jdmulloy · 2020-03-10T22:06:30Z

I'm seeing this too. Even adding the new configuration, it still shows a change, even when there isn't. For example.

     - grant {
          - permissions = [
              - "READ_ACP",
              - "WRITE",
            ] -> null
          - type        = "Group" -> null
          - uri         = "http://acs.amazonaws.com/groups/s3/LogDelivery" -> null
        }
      - grant {
          - id          = "REDACTED1" -> null
          - permissions = [
              - "FULL_CONTROL",
            ] -> null
          - type        = "CanonicalUser" -> null
        }
      + grant {
          + id          = "REDACTED1"
          + permissions = [
              + "FULL_CONTROL",
            ]
          + type        = "CanonicalUser"
        }
      + grant {
          + permissions = [
              + "READ_ACP",
              + "WRITE",
            ]
          + type        = "Group"
          + uri         = "http://acs.amazonaws.com/groups/s3/LogDelivery"
        }

jdmulloy · 2020-03-10T23:07:00Z

Actually it works when you have all the rules in terraform. I hadn't done the last one yet.

ewbankkit · 2020-03-11T11:30:32Z

Maybe due to the merge of #3728 - S3 bucket ACL grants are now managed by Terraform.

timvisher · 2020-03-11T11:55:29Z

@ewbankkit I think I've seen in the past where terraform can manage something but hasn't actually been managing it where the provider will silently ignore the diff but maybe that's a false memory.

Still, this kind of change feels odd for a minor version bump since I would expect those to be non-breaking.

tomelliff · 2020-03-11T16:10:47Z

Does this need a state migration adding?

aywrite · 2020-03-11T23:53:41Z

It's the fact that terraform wants to remove a grant which is created by default whenever a bucket is created that I find confusing - IMO if the grants block isn't specified then it should leave the default grant alone.

When you create a bucket or an object, Amazon S3 creates a default ACL that grants the resource owner full control over the resource.
https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html

Generated plan

      - grant {
          - id          = "<creatorid>" -> null
          - permissions = [
              - "FULL_CONTROL",
            ] -> null
          - type        = "CanonicalUser" -> null
        }

andypoole · 2020-03-16T10:59:23Z

I've also seen this today and while adding the grant to our code to mitigate the issue I also noticed that the documentation of the grant {} block that was added with change #3728 is incorrect:

https://github.com/terraform-providers/terraform-provider-aws/pull/3728/files#diff-7f5ed2626ccd023dd9d0f679c2526b6fR323
https://github.com/terraform-providers/terraform-provider-aws/pull/3728/files#diff-7f5ed2626ccd023dd9d0f679c2526b6fR328

These lines in the provided example code use permission instead of permissions and also FULL_ACCESS where the actual correct value should be FULL_CONTROL (The former fails to plan as an invalid value).

https://github.com/terraform-providers/terraform-provider-aws/pull/3728/files#diff-7f5ed2626ccd023dd9d0f679c2526b6fR488

This line also lists FULL_ACCESS as a valid option instead of FULL_CONTROL.

Chhed13 · 2020-03-20T07:28:30Z

I reproduced it on 2.52 version solo. That is not a migration bug, that some bug in grant state calculation.
To reproduce:

create simple bucket:

resource "aws_s3_bucket" "XXX" {
  bucket = ...
}

change smth in grants. For instance
support your changes in tf

resource "aws_s3_bucket" "XXX" {
  bucket = ...
  grant {
      id = "${data.aws_canonical_user_id.current.id}"
      type = "CanonicalUser"
      permissions = ["FULL_CONTROL"]
  }
  grant {
      .....<your changes>
  }
}

If you reproduce it 100% matched including ordering - there will be no diff
If not - you'll get full recreation. After recreation that will be ok.

I see that logic of creation grant is correct and it looks like a sorting issue inside terraform state. The ordering issue looks strange because storage of grant block made on hashes, not lists. But that looks like most true story.

mrliptontea · 2020-05-27T15:49:49Z

Hi,

I'm also seeing an issue with how grant is calculated for the plan. Tested on 2.63.

Create a bucket with grant:

resource "aws_s3_bucket" "my-example-terraform-grant" {
    bucket = "my-example-terraform-grant"

    grant {
        id          = data.aws_canonical_user_id.current_user.id
        type        = "CanonicalUser"
        permissions = ["FULL_CONTROL"]
    }
}

terraform apply.

terraform plan yields an in-place update to add the grant.

Output of terraform plan

  # aws_s3_bucket.my-example-terraform-grant will be updated in-place
  ~ resource "aws_s3_bucket" "my-example-terraform-grant" {
        acl                         = "private"
        arn                         = "arn:aws:s3:::my-example-terraform-grant"
        bucket                      = "my-example-terraform-grant"
        bucket_domain_name          = "my-example-terraform-grant.s3.amazonaws.com"
        bucket_regional_domain_name = "my-example-terraform-grant.s3.eu-west-1.amazonaws.com"
        force_destroy               = false
        hosted_zone_id              = "XXXXXXXXXXXXXX"
        id                          = "my-example-terraform-grant"
        region                      = "eu-west-1"
        request_payer               = "BucketOwner"
        tags                        = {}

      + grant {
          + id          = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
          + permissions = [
              + "FULL_CONTROL",
            ]
          + type        = "CanonicalUser"
        }

        versioning {
            enabled    = false
            mfa_delete = false
        }
    }

Plan: 0 to add, 1 to change, 0 to destroy.

At this point I could terraform apply forever, it will always see a change.

This is the bucket in the backend state

{
  "mode": "managed",
  "type": "aws_s3_bucket",
  "name": "my-example-terraform-grant",
  "provider": "provider.aws",
  "instances": [
    {
      "schema_version": 0,
      "attributes": {
        "acceleration_status": "",
        "acl": "private",
        "arn": "arn:aws:s3:::my-example-terraform-grant",
        "bucket": "my-example-terraform-grant",
        "bucket_domain_name": "my-example-terraform-grant.s3.amazonaws.com",
        "bucket_prefix": null,
        "bucket_regional_domain_name": "my-example-terraform-grant.s3.eu-west-1.amazonaws.com",
        "cors_rule": [],
        "force_destroy": false,
        "grant": [
          {
            "id": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
            "permissions": [
              "FULL_CONTROL"
            ],
            "type": "CanonicalUser",
            "uri": ""
          }
        ],
        "hosted_zone_id": "XXXXXXXXXXXXXX",
        "id": "my-example-terraform-grant",
        "lifecycle_rule": [],
        "logging": [],
        "object_lock_configuration": [],
        "policy": null,
        "region": "eu-west-1",
        "replication_configuration": [],
        "request_payer": "BucketOwner",
        "server_side_encryption_configuration": [],
        "tags": {},
        "versioning": [
          {
            "enabled": false,
            "mfa_delete": false
          }
        ],
        "website": [],
        "website_domain": null,
        "website_endpoint": null
      },
      "private": "bnVsbA=="
    }
  ]
}

so0k · 2020-07-07T07:27:29Z

still having the same issue with provider.aws ~> 2.69 and terraform 0.12.25

[terragrunt] 2020/07/07 15:27:01 Running command: terraform providers
.
├── provider.aws ~> 2.69
├── module.bucket_access_policies
│   └── provider.aws (inherited)
├── module.bucket_policies
│   └── provider.aws (inherited)
├── module.s3_bucket
│   └── provider.aws (inherited)
└── module.system_users
    └── provider.aws (inherited)

rahulk94 · 2020-07-14T23:59:43Z

I've tried to workaround this issue by having an explicit grant declaration like @mrliptontea above in #12332 (comment), but even if I pin my terraform-aws-provider version to 2.53 (lowest I can go in my project) I still get prompted to create a new policy every time I run terraform plan.

provider "aws" {
  region  = var.aws_region
  version = "<= 2.53"

so it sounds like perhaps an upstream change is causing issues.

# versions for reference
$ terraform -v 
Terraform v0.12.28
+ provider.archive v1.3.0
+ provider.aws v2.53.0
+ provider.template v2.1.2

mrliptontea · 2021-02-02T22:38:31Z

Any movement on this issue?

This is blocking us from going beyond 2.51. In fact, this is still an issue in 3.26. Did anyone had any luck using grants at all?

misilot · 2021-05-31T18:43:04Z

This still seems to be a problem with v3.42.0, is there any thoughts on how to resolve this?

quinot · 2021-07-15T14:22:59Z

Hi,

I'm also seeing an issue with how grant is calculated for the plan. Tested on 2.63.

Create a bucket with grant:

resource "aws_s3_bucket" "my-example-terraform-grant" {
    bucket = "my-example-terraform-grant"

    grant {
        id          = data.aws_canonical_user_id.current_user.id
        type        = "CanonicalUser"
        permissions = ["FULL_CONTROL"]
    }
}

terraform apply.

terraform plan yields an in-place update to add the grant.
Output of terraform plan

  # aws_s3_bucket.my-example-terraform-grant will be updated in-place
  ~ resource "aws_s3_bucket" "my-example-terraform-grant" {
        acl                         = "private"
        arn                         = "arn:aws:s3:::my-example-terraform-grant"
        bucket                      = "my-example-terraform-grant"
        bucket_domain_name          = "my-example-terraform-grant.s3.amazonaws.com"
        bucket_regional_domain_name = "my-example-terraform-grant.s3.eu-west-1.amazonaws.com"
        force_destroy               = false
        hosted_zone_id              = "XXXXXXXXXXXXXX"
        id                          = "my-example-terraform-grant"
        region                      = "eu-west-1"
        request_payer               = "BucketOwner"
        tags                        = {}

      + grant {
          + id          = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
          + permissions = [
              + "FULL_CONTROL",
            ]
          + type        = "CanonicalUser"
        }

        versioning {
            enabled    = false
            mfa_delete = false
        }
    }

Plan: 0 to add, 1 to change, 0 to destroy.

At this point I could terraform apply forever, it will always see a change.

This is the bucket in the backend state

Should there be a separate issue for this bug?

mrliptontea · 2021-07-16T15:16:41Z

Should there be a separate issue for this bug?

I was under the impression that that's the main issue? Or a part of it at least.

justinretzolk · 2024-03-07T21:36:51Z

Hey all 👋 Given the length of time its been since anyone reported this, and that it appears to have affected versions of the provider that are no longer supported, I'm going to close this issue out. If you experience similar issues on more recent versions of the provider, please open a new issue, referencing this one where necessary, so that we can triage it effectively.

github-actions · 2024-04-07T02:04:48Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

ghost added service/iam Issues and PRs that pertain to the iam service. service/s3 Issues and PRs that pertain to the s3 service. labels Mar 10, 2020

github-actions bot added the needs-triage Waiting for first response or review from a maintainer. label Mar 10, 2020

antonbabenko mentioned this issue Mar 13, 2020

Feature Request - S3 ACL Policy Grant terraform-aws-modules/terraform-aws-s3-bucket#20

Closed

dooreelko mentioned this issue Jun 12, 2020

data "aws_cloudformation_stack" fails to use correct provider #13741

Closed

breathingdust added bug Addresses a defect in current functionality. and removed needs-triage Waiting for first response or review from a maintainer. labels Jul 10, 2020

justinretzolk closed this as completed Mar 7, 2024

github-actions bot locked as resolved and limited conversation to collaborators Apr 7, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Breaking change for several resources from 2.51.0 → 2.52.0 #12332

Breaking change for several resources from 2.51.0 → 2.52.0 #12332

timvisher commented Mar 10, 2020

jdmulloy commented Mar 10, 2020 •

edited

jdmulloy commented Mar 10, 2020

ewbankkit commented Mar 11, 2020

timvisher commented Mar 11, 2020

tomelliff commented Mar 11, 2020

aywrite commented Mar 11, 2020

andypoole commented Mar 16, 2020 •

edited

Chhed13 commented Mar 20, 2020 •

edited

mrliptontea commented May 27, 2020

so0k commented Jul 7, 2020 •

edited

rahulk94 commented Jul 14, 2020 •

edited

mrliptontea commented Feb 2, 2021

misilot commented May 31, 2021

quinot commented Jul 15, 2021

mrliptontea commented Jul 16, 2021

justinretzolk commented Mar 7, 2024

github-actions bot commented Apr 7, 2024

Breaking change for several resources from 2.51.0 → 2.52.0 #12332

Breaking change for several resources from 2.51.0 → 2.52.0 #12332

Comments

timvisher commented Mar 10, 2020

Community Note

Terraform Versions

Affected Resource(s)

Terraform Configuration Files

Expected Behavior

Actual Behavior

Steps to Reproduce

jdmulloy commented Mar 10, 2020 • edited

jdmulloy commented Mar 10, 2020

ewbankkit commented Mar 11, 2020

timvisher commented Mar 11, 2020

tomelliff commented Mar 11, 2020

aywrite commented Mar 11, 2020

andypoole commented Mar 16, 2020 • edited

Chhed13 commented Mar 20, 2020 • edited

mrliptontea commented May 27, 2020

so0k commented Jul 7, 2020 • edited

rahulk94 commented Jul 14, 2020 • edited

mrliptontea commented Feb 2, 2021

misilot commented May 31, 2021

quinot commented Jul 15, 2021

mrliptontea commented Jul 16, 2021

justinretzolk commented Mar 7, 2024

github-actions bot commented Apr 7, 2024

jdmulloy commented Mar 10, 2020 •

edited

andypoole commented Mar 16, 2020 •

edited

Chhed13 commented Mar 20, 2020 •

edited

so0k commented Jul 7, 2020 •

edited

rahulk94 commented Jul 14, 2020 •

edited