google_service_account: Service account gets created, but apply run fails with "Error reading service account after creation"

[neurolabs]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

$ terraform -v
Terraform v0.12.28
+ provider.google v3.28.0
...

Affected Resource(s)

• google_service_account

Terraform Configuration Files

resource "google_service_account" "foobar" {
  project      = google_project.main.project_id
  account_id   = "foobar"
  display_name = "Foo Bar"

  provisioner "local-exec" {
    # Creation of service accounts is eventually consistent:
    # https://www.terraform.io/docs/providers/google/r/google_service_account.html
    command = "sleep 10"
  }
}

Debug Output

None given

Panic Output

Expected Behavior

The account is created successfully and google_service_account resource finishes successfully.

Actual Behavior

The account is created successfully (eventually) and google_service_account resource fails.

Error: Error reading service account after creation: googleapi: Error 404: Service account projects/main-3e/serviceAccounts/foobar@main-3e.iam.gserviceaccount.com does not exist., notFound
  on project-api.tf line 41, in resource "google_service_account" "foobar":
  41: resource "google_service_account" "foobar" {

Steps to Reproduce

1. terraform apply

Important Factoids

References

• Service Account resource gets created but not added to state #6362
  • @c2thorn did a pull request Add retry to service account creation GoogleCloudPlatform/magic-modules#3513 that was supposed to improve the situation, which does not seem to be enough.

[c2thorn]

Hey @neurolabs sorry to hear you are still getting this error. The PR added a retry timeout that is adjustable via the create timeout. It's possible that your service account creation is taking longer than the default timeout of 5 minutes to become eventually consistent.

I'm unable to get a failure on apply in my repro, do you have debug logs for me to look through?