version 0.66.4 breaks if we use policies that are not created before using it.

[giom-l]

Describe the Bug

If a policy ARN is provided as task_exec_policy_arns and this policy is created at the same time the module is, then :

• it will work with module in version 0.66.3
• it will fail with version 0.66.4 (since the switch from count to for_each)

The error returned by version 0.66.4 is

│ Error: Invalid for_each argument
│ 
│   on .terraform/modules/service-task/main.tf line 289, in resource "aws_iam_role_policy_attachment" "ecs_exec":
│  289:   for_each   = local.create_exec_role ? toset(var.task_exec_policy_arns) : toset([])
│     ├────────────────
│     │ local.create_exec_role is true
│     │ var.task_exec_policy_arns is list of string with 1 element
│ 
│ The "for_each" set includes values derived from resource attributes that
│ cannot be determined until apply, and so Terraform cannot determine the
│ full set of keys that will identify the instances of this resource.
│ 
│ When working with unknown values in for_each, it's better to use a map
│ value where the keys are defined statically in your configuration and where
│ only the values contain apply-time results.
│ 
│ Alternatively, you could use the -target planning option to first apply
│ only the resources that the for_each value depends on, and then apply a
│ second time to fully converge.

Adding the s3 policy in depends_on didn't help.

Expected Behavior

A minor version should not introduces breaking changes.
The provided code should still work.

Note we can workaround the issue by :

• Create the policy before running the entire module ==> not very automation friendly
• Manually handle all the role creation part and only provide the ARNs to the module (tested and approved)

Steps to Reproduce

Here is a sample code (dummy policy but sufficient to demonstrate) that is run successfully in 0.66.3 but fails in 0.66.4.
It may be possible to adapt it and create the policy before running service-task module but that's not very automation-friendly.

terraform {
  # Not tested with version 1.x < 1.2.0 but should be compatible.
  required_version = ">= 1.2.0, < 2.0.0"
  backend "s3" {}
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      # Switch to 4.x provider version, confident that there are no breaking change until 5.x
      version = ">= 4.0, < 5.0"
    }
  }
}

provider "aws" {
  region = var.region
}

variable "region" {
  type = string
}
variable "private_subnet_ids" {
  type = list(string)
}

variable "vpc_id" {
  type = string
}

variable "security_group_id" {
  type = string
}


data "aws_caller_identity" "caller" {}
# ------------------------------------
# ----- Cloudwatch               -----
# ------------------------------------
resource "aws_cloudwatch_log_group" "test" {
  name              = "test-service"
  retention_in_days = 1
}

# ------------------------------------
# ----- Cluster               -----
# ------------------------------------
resource "aws_ecs_cluster" "cluster" {
  name = "test-cluster"
}

resource "aws_ecs_cluster_capacity_providers" "cluster" {
  cluster_name       = aws_ecs_cluster.cluster.name
  capacity_providers = ["FARGATE", "FARGATE_SPOT"]
  default_capacity_provider_strategy {
    capacity_provider = "FARGATE_SPOT"
  }
}


# ------------------------------------------------
# ----- Permissions specific task exec role  -----
# ------------------------------------------------
data "aws_iam_policy_document" "s3" {
  statement {
    sid    = "AllowKMSKeyUsage"
    effect = "Allow"
    actions = [
      "kms:Decrypt"
    ]
    resources = [
      "arn:aws:kms:::whatever-key"
    ]
  }
  statement {
    sid    = "AllowListBucketForConfigsBucket"
    effect = "Allow"
    actions = [
      "s3:ListBucket"
    ]
    resources = [
      "arn:aws:s3:::whatever-bucket"
    ]
    condition {
      test     = "StringLike"
      variable = "s3:prefix"
      values = [
        "test/*",
      ]
    }
  }
  statement {
    sid    = "AllowReadonlyConfigObjects"
    effect = "Allow"
    actions = [
      "s3:GetObject"
    ]
    resources = [
      "arn:aws:s3:::whatever-bucket/test/*"
    ]
  }
}
resource "aws_iam_policy" "s3" {
  name   = "test-service-s3-access"
  policy = data.aws_iam_policy_document.s3.json
}

# -------------------------------------------
# ----- ECS Tasks containers definition -----
# -------------------------------------------
module "nginx-container-definition" {
  source          = "cloudposse/ecs-container-definition/aws"
  version         = "0.58.1"
  container_name  = "nginx"
  container_image = "public.ecr.aws/nginx/nginx:1.23.3"
  essential       = true
  port_mappings = [
    {
      containerPort = 80
      hostPort      = 80
      protocol      = "tcp"
    }
  ]
  log_configuration = {
    logDriver = "awslogs",
    options = {
      awslogs-group         = aws_cloudwatch_log_group.test.name,
      awslogs-region        = var.region,
      awslogs-stream-prefix = "nginx"
    }
  }
  stop_timeout = 120
}

# -------------------------------------------
# ----- ECS Service definition -----
# -------------------------------------------
module "api-service-task" {
  source                    = "cloudposse/ecs-alb-service-task/aws"
  version                   = "0.66.3"
  name                      = "test-service"
  ecs_cluster_arn           = aws_ecs_cluster.cluster.arn
  container_definition_json = module.nginx-container-definition.json_map_encoded_list
  launch_type               = "FARGATE"
  exec_enabled              = true

  // Specific policy to access secrets when starting the container
  task_exec_policy_arns = [aws_iam_policy.s3.arn]

  vpc_id             = var.vpc_id
  security_group_ids = [var.security_group_id]
  subnet_ids         = var.private_subnet_ids
  network_mode       = "awsvpc"
  assign_public_ip   = false

  desired_count                  = 1
  task_memory                    = 512
  task_cpu                       = 256
  ignore_changes_task_definition = false
}

Steps to reproduce the behavior:
1 - terraform init
2 - terraform apply
3 - Ensure everything is ok
4 - terraform destroy (required, otherwise the policy would exists and everything would be running fine)
5 - Change the api-service-task module version from 0.66.3 to 0.66.4
6 - terraform init --upgrade && terraform apply