Race condition between state machine role creation and use. Long time to propagate the role.

[vishalGitAcc]

Terraform Version
Terraform version 2.1.0

Problem: The state machine exec role takes time to propagate and an explicit “depends on” constraint does not help.

Terraform Configuration Files

data "aws_iam_policy_document" "states_exec_role_document" {
  statement {
    actions = ["sts:AssumeRole"]

    principals {
      type        = "Service"
      identifiers = ["states.amazonaws.com"]
    }
  }
}

resource "aws_iam_role" "states_exec_role" {
  name               = "${local.stack_name}-${local.pod}-statesExecRole"
  assume_role_policy = "${data.aws_iam_policy_document.states_exec_role_document.json}"
}

resource "aws_sfn_state_machine" "state_machine" {
 depends_on = ["aws_iam_role.states_exec_role"]
 name       = "${local.stack_name}-${local.pod}-state-machine"
 definition = "${data.template_file.definition.rendered}"
 role_arn   = "${aws_iam_role.states_exec_role.arn}"
}

Crash Output
"* aws_sfn_state_machine.state_machine: Error creating Step Function State Machine: AccessDeniedException: Neither the global service principal states.amazonaws.com, nor the regional one is authorized to assume the provided role.
status code: 400, request id: b26b3cc8-442d-11e9-83bb-03aa166dd894"

Expected Behavior
Apply complete! Resources: X added, 0 changed, 0 destroyed.

Actual Behavior
Fails to apply because iam roles takes time to propagate across regions. Apply succeeds on second application. It takes a few seconds for the IAM role to be available.

Steps to Reproduce
terraform apply

Additional context

Following hack addresses the problem, but this is still a hack.

resource "null_resource" "delay" {
 provisioner "local-exec" {
   command = "sleep 30"
 }
 triggers = {
   "states_exec_role" = "${aws_iam_role.states_exec_role.arn}"
 }
}

resource "aws_sfn_state_machine" "state_machine" {
 name       = "${local.stack_name}-${local.pod}-state-machine"
 definition = "${data.template_file.definition.rendered}"
 role_arn   = "${aws_iam_role.states_exec_role.arn}"
 depends_on = ["null_resource.delay"]
}

Looking for a better solution to address the race condition.

[sstoeckel]

I got an equal issue. The Problem is the State Machine will be created before the IAM Role and IAM Role Policy Attachment.

[mauza]

Experiencing the same issue. I've built a few step functions in the past and never ran into this...

[rbundy]

Experiencing the same issue

[prokvk]

same issue here

[Vilaggio]

Same issue here. I attempted to add a trust relationship but that error continues...

I created an IAM role that allows FULL access to every AWS service I'm using.

[Vilaggio]

I fixed mine by going back and adding the correct IAM role in the state machine.

[bflad]

The fix to allow the aws_sfn_state_machine resource to retry IAM Role associated failures on creation has been merged and will release with version 2.69.0 of the Terraform AWS Provider, likely tomorrow. Thanks to @DrFaust92 for the implementation. 👍

[ghost]

This has been released in version 2.69.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!