Tainted resource creation

[Skyfallz]

Hi there,

I wonder if there is any way to taint a resource on its creation ? For now I create resources and taint them in my CI/CD pipeline, but I'd like to set it during the Terraform run (seems cleaner to me, and there is no need to update my CD pipeline when I add a new resource).

Current Terraform Version

0.12

Use-cases

In order to prevent my ALB routing rules to be mixed up (after a create/delete replacement, for example), I create them with depends_on meta-argument, set up "create_before_destroy" lifecycle argument on each resource (to avoid downtime on create/delete replacement), and taint them afterwards to ensure my set of rules is in the right order after each run. I can't use the 'priority' argument, becase this ALB is shared between all of my integration environments, and each set of rule has its 'relative' order (so I can't use an 'absolute' order). This works as intented, but I'd like to define my tainting inside my TF instead.

Attempted Solutions

resource "aws_lb_listener_rule" "ms-xxx-1" {
  listener_arn = "${data.terraform_remote_state.infra.outputs.aws_lb_listener_fargate_alb_https_arn}"

  action {
    type             = "forward"
    target_group_arn = "${aws_lb_target_group.ms-xxx.arn}"
  }

  condition {
    field  = "path-pattern"
    values = ["/xxx/myhealthcheck]
  }
  condition {
    field  = "host-header"
    values = var.account == "prod" ? ["app.xxx.com"] : ["${var.env}.xxx.com"]
  }

  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_lb_listener_rule" "ms-xxx-2" {
  listener_arn = "${data.terraform_remote_state.infra.outputs.aws_lb_listener_fargate_alb_https_arn}"

  action {
    type             = "forward"
    target_group_arn = "${aws_lb_target_group.ms-xxx.arn}"
  }

  condition {
    field  = "path-pattern"
    values = ["/xxx/publicpages*"]
  }
  condition {
    field  = "host-header"
    values = var.account == "prod" ? ["app.xxx.fr"] : ["${var.env}.xxx.com"]
  }

  lifecycle {
    create_before_destroy = true
  }

  depends_on = [aws_lb_listener_rule.ms-xxx-1]
}

resource "aws_lb_listener_rule" "ms-xxx-3" {
  listener_arn = "${data.terraform_remote_state.infra.outputs.aws_lb_listener_fargate_alb_https_arn}"

  action {
    type             = "forward"
    target_group_arn = "${aws_lb_target_group.ms-xxx.arn}"
  }

  condition {
    path_pattern {
      values = ["/microservice_prefix*"]
    }
  }

  condition {
    host_header {
      values = var.account == "prod" ? ["app.xxx.com"] : ["${var.env}.xxx.com"]
    }
  }

  condition {
    http_header {
      http_header_name = "Authorization"
      values           = ["Bearer *"]
    }
  }

  lifecycle {
    create_before_destroy = true
  }

  depends_on = [aws_lb_listener_rule.ms-xxx-2]
}

resource "aws_lb_listener_rule" "ms-xxx-4" {
  listener_arn = "${data.terraform_remote_state.infra.outputs.aws_lb_listener_fargate_alb_https_arn}"

  action {
    type = "authenticate-cognito"

    authenticate_cognito {
      user_pool_arn       = "arn:aws:cognito-idp:${var.region}:${myaccountID}:userpool/${data.terraform_remote_state.infra.outputs.aws_cognito_user_pool_cognito_id}"
      user_pool_client_id = "${data.terraform_remote_state.env.outputs.aws_cognito_user_pool_client_cognito_id}"

      user_pool_domain = "${myuserpooldomain}"

      scope                      = "openid"
      session_cookie_name        = "AWSELBAuthSessionCookie"
      session_timeout            = "604800"
      on_unauthenticated_request = "authenticate"
    }
  }

  action {
    type             = "forward"
    target_group_arn = "${aws_lb_target_group.ms-xxx.arn}"
  }

  condition {
    field  = "path-pattern"
    values = ["/microservice_prefix*"]
  }
  condition {
    field  = "host-header"
    values = var.account == "prod" ? ["app.xxx.com"] : ["${var.env}.xxx.com"]
  }

  lifecycle {
    create_before_destroy = true
  }

  depends_on = [aws_lb_listener_rule.ms-xxx-3]
}

And the in my CD pipeline :

    - terraform apply -lock-timeout=300s -var-file "vars/${TF_ACCOUNT}.tfvars" -var "account=${TF_ACCOUNT}" -var "env=${GIT_BRANCH_SHORT}" -var "external_account_id=${TF_ACC_ID}" -var "region=${AWS_REGION}" --auto-approve
    - for i in `seq 1 4`;do terraform taint -lock=true -lock-timeout=300s aws_lb_listener_rule.ms-xxx-$i;done

It works, but IMO the loop part in the CD pipeline is not a really clean and mistake-proof solution (someone could add a listener_rule and forget to taint it, which would eventually lead to mix the whole order of my rule sequence).

Proposal

If no way of tainting resources within my Terraform code is available, would it seem feasable/a good idea to implement a new meta-argument on the lifecycle block ? Something like :

resource "aws_lb_listener_rule" "ms-xxx-1" {
  listener_arn = "${data.terraform_remote_state.infra.outputs.aws_lb_listener_fargate_alb_https_arn}"

  action {
    type             = "forward"
    target_group_arn = "${aws_lb_target_group.ms-xxx.arn}"
  }

  condition {
    field  = "path-pattern"
    values = ["/xxx/myhealthcheck]
  }
  condition {
    field  = "host-header"
    values = var.account == "prod" ? ["app.xxx.com"] : ["${var.env}.xxx.com"]
  }

  lifecycle {
    create_before_destroy = true
    tainted                          = true
  }
}

Thanks for your time, let me know if I wasn't clear enough or if you need more info about this usecase or anything.

Regards,

[teamterraform]

Hi @Skyfallz! Thanks for sharing this use-case.

This doesn't really fit within the intended use of terraform taint, so while we will label this issue and keep it open for future consideration we also want to be forthcoming that we're likely to prefer to find a different solution to your problem instead, because the "taint" mechanism is not supposed to be part of any routine workflow.

Since your motivation here is to work around some problems caused by how AWS models load balancer listener rules, we'd suggest also opening an issue about this in the AWS provider repository if you haven't already. The ideal way to address this AWS-specific problem would be via some feature in the AWS provider, but the AWS provider team is in a better position to explore and evaluate solutions due to their stronger familiarity with the AWS API.

---

The Terraform Core team isn't super familiar with this particular resource type in the AWS provider, but we can see that priority is implemented such that if you don't set it the provider will export the priority value that was automatically selected, which suggests a possible workaround for you in the meantime:

resource "aws_lb_listener_rule" "ms-xxx-1" {
  listener_arn = "${data.terraform_remote_state.infra.outputs.aws_lb_listener_fargate_alb_https_arn}"

  # Priority for this first rule will be chosen automatically by the
  # ALB API. The other rules in this module will be assigned
  # subsequent consecutive priorities.

  action {
    type             = "forward"
    target_group_arn = "${aws_lb_target_group.ms-xxx.arn}"
  }

  condition {
    field  = "path-pattern"
    values = ["/xxx/myhealthcheck"]
  }
  condition {
    field  = "host-header"
    values = var.account == "prod" ? ["app.xxx.com"] : ["${var.env}.xxx.com"]
  }
}

resource "aws_lb_listener_rule" "ms-xxx-2" {
  listener_arn = "${data.terraform_remote_state.infra.outputs.aws_lb_listener_fargate_alb_https_arn}"

  # Rule 2 is the next highest priority after rule 1.
  priority = aws_lb_listener_rule.ms-xxx-1.priority + 1

  action {
    type             = "forward"
    target_group_arn = "${aws_lb_target_group.ms-xxx.arn}"
  }

  condition {
    field  = "path-pattern"
    values = ["/xxx/publicpages*"]
  }
  condition {
    field  = "host-header"
    values = var.account == "prod" ? ["app.xxx.fr"] : ["${var.env}.xxx.com"]
  }

  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_lb_listener_rule" "ms-xxx-3" {
  listener_arn = "${data.terraform_remote_state.infra.outputs.aws_lb_listener_fargate_alb_https_arn}"

  # Rule 3 is the next highest priority after rule 2.
  priority = aws_lb_listener_rule.ms-xxx-2.priority + 1

  action {
    type             = "forward"
    target_group_arn = "${aws_lb_target_group.ms-xxx.arn}"
  }

  condition {
    path_pattern {
      values = ["/microservice_prefix*"]
    }
  }

  condition {
    host_header {
      values = var.account == "prod" ? ["app.xxx.com"] : ["${var.env}.xxx.com"]
    }
  }

  condition {
    http_header {
      http_header_name = "Authorization"
      values           = ["Bearer *"]
    }
  }

  lifecycle {
    create_before_destroy = true
  }
}

# etc, etc ...

In the above example, we let the first rule have its priority automatically selected as before, but then all of the other rules will occupy successive priority values automatically calculated based on the first. The subsequent rules will be created sequentially, each one using the priority value selected for the previous to derive its own priority value.

A challenge with this approach is that if you need to add a new rule in the future then you'll need to reassign all subsequent ids, so this is clearly not a perfect answer but at least it requires additional manual steps only in the case where a new rule is being added, and not whenever an existing rule replaced due to configuration changes.