collection functions on maps of for_each resources are evaluated before changes to the input for for_each

[tabacco]

Terraform Version

0.12.13

Terraform Configuration Files

First Run

resource "aws_secretsmanager_secret" "test" {
  for_each = {
    "secret-one" : "cool first secret",
    "secret-two" : "cool second secret",
  }

  name        = each.key
  description = each.value
}

data "aws_iam_policy_document" "test-policy" {
  statement {
    actions   = ["secretsmanager:GetSecretValue"]
    resources = values(aws_secretsmanager_secret.test)[*].arn
  }
}

resource "aws_iam_policy" "test-policy" {
  policy = data.aws_iam_policy_document.test-policy.json
}

Second and Third Runs

resource "aws_secretsmanager_secret" "test" {
  for_each = {
    "secret-one" : "cool first secret",
  }

  name        = each.key
  description = each.value
}

data "aws_iam_policy_document" "test-policy" {
  statement {
    actions   = ["secretsmanager:GetSecretValue"]
    resources = values(aws_secretsmanager_secret.test)[*].arn
  }
}

resource "aws_iam_policy" "test-policy" {
  policy = data.aws_iam_policy_document.test-policy.json
}

Debug Output

First run: https://gist.github.com/tabacco/cc621d82e8fad758402fe8fcdc3d708d
Second run: https://gist.github.com/tabacco/00cfd84fec5b3e058b40906ffdb7dabb
Third run: https://gist.github.com/tabacco/3b06c75f99cc46e53882027ad198b97e

Expected Behavior

On the second apply, I would expect "secret-two" to be deleted, and the policy json to be updated to remove "secret-two"

Actual Behavior

On the second apply "secret-two" is deleted but the policy json is unchanged. On the third apply the policy json is updated correctly.

These are the plans from the second and third runs (I've omitted the first, since it works as expected):

Second Run

  - resource "aws_secretsmanager_secret" "test" {
      - arn                     = "arn:aws:secretsmanager:us-west-1:000000000000:secret:secret-two-ZlwsrV" -> null
      - description             = "cool second secret" -> null
      - id                      = "arn:aws:secretsmanager:us-west-1:000000000000:secret:secret-two-ZlwsrV" -> null
      - name                    = "secret-two" -> null
      - recovery_window_in_days = 30 -> null
      - rotation_enabled        = false -> null
      - tags                    = {} -> null
    }

Plan: 0 to add, 0 to change, 1 to destroy.

Third Run

  ~ resource "aws_iam_policy" "test-policy" {
        arn    = "arn:aws:iam::000000000000:policy/terraform-20191115161139914600000001"
        id     = "arn:aws:iam::000000000000:policy/terraform-20191115161139914600000001"
        name   = "terraform-20191115161139914600000001"
        path   = "/"
      ~ policy = jsonencode(
          ~ {
              ~ Statement = [
                  ~ {
                        Action   = "secretsmanager:GetSecretValue"
                        Effect   = "Allow"
                      ~ Resource = [
                          - "arn:aws:secretsmanager:us-west-1:000000000000:secret:secret-two-ZlwsrV",
                          - "arn:aws:secretsmanager:us-west-1:000000000000:secret:secret-one-JjgnDU",
                        ] -> "arn:aws:secretsmanager:us-west-1:000000000000:secret:secret-one-JjgnDU"
                        Sid      = ""
                    },
                ]
                Version   = "2012-10-17"
            }
        )
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Steps to Reproduce

1. Use config from "First Run" above.
2. terraform init
3. terraform apply
4. Update config to "Second and Third Runs" above
5. terraform apply
6. terraform apply

Additional Context

Nothing interesting to report, just vanilla terraform with for_each.

References

[tabacco]

Apologies for the awkward title, I struggled to figure out how to summarize this. Please feel free to rename!

[teamterraform]

Hi @tabacco! Thanks for reporting this.

It looks like you've found a specific case here of the general problem described in #17034 (under "When Data Resource Arguments change"). Terraform is unfortunately behaving as designed here -- refreshing always happens before taking into account changes to resource configurations -- but that other issue is a proposal to change that design to perform both refreshing and planning at once so that the refresh actions (which include reading data sources) can take into account changes to resource instances that we plan to make during the apply step.

Since we have #17034 open tracking the proposal we're going to close this just to consolidate the discussion, but this extensive writeup will make a good test case to evaluate future design iterations and a final implementation of #17034 against, so we very much appreciate you taking the time to write it up in detail!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.