[Bug]: aws_elasticache_replication_group is delaying terraform plan for 30 seconds, presumably waiting for the group to become available even though the group is already available

[stefansundin]

Terraform Core Version

1.3.9

AWS Provider Version

4.61.0

Affected Resource(s)

aws_elasticache_replication_group

Expected Behavior

The resource shouldn't needlessly slow down terraform plan or terraform apply. If a replication group is created and available then it shouldn't need to wait.

Actual Behavior

If you have a aws_elasticache_replication_group created, the provider will wait for 30 seconds before it is done refreshing the state for the resource. According to the logs, it is waiting for the group state to become available. But it will wait for 30 seconds even if the group is already available. It is a very annoying delay every time you want to simply plan a change.

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

resource "aws_elasticache_replication_group" "redis" {
  replication_group_id       = "tftest-redis"
  description                = "Managed by Terraform"
  node_type                  = "cache.t3.micro"
  subnet_group_name          = "test"
  engine_version             = "6.2"
  port                       = 6379
  num_cache_clusters         = 1
  multi_az_enabled           = false
  automatic_failover_enabled = false
  apply_immediately          = true
}

Steps to Reproduce

First create the replication group:

$ terraform apply

Then, without making any changes, just plan and wait for ~35 seconds:

$ time terraform plan

[...]

real	0m35.135s
user	0m1.437s
sys	0m0.362s

Debug Output

Here's some output from TF_LOG=1 terraform plan:

2023-04-02T14:31:04.489-0700 [DEBUG] provider.terraform-provider-aws: [DEBUG] ElastiCache Replication Group (tftest-redis) Configuration Endpoint is nil
2023-04-02T14:31:04.489-0700 [DEBUG] provider.terraform-provider-aws: [DEBUG] ElastiCache Replication Group (tftest-redis) Primary Endpoint is not nil
2023-04-02T14:31:04.489-0700 [DEBUG] provider.terraform-provider-aws: [DEBUG] Waiting for ElastiCache Replication Group (tftest-redis) to become available
2023-04-02T14:31:04.489-0700 [DEBUG] provider.terraform-provider-aws: [DEBUG] Waiting for state to become: [available]
2023-04-02T14:31:05.324-0700 [TRACE] dag/walk: vertex "provider[\"registry.terraform.io/hashicorp/aws\"] (close)" is waiting for "aws_elasticache_replication_group.redis (expand)"
2023-04-02T14:31:05.324-0700 [TRACE] dag/walk: vertex "root" is waiting for "provider[\"registry.terraform.io/hashicorp/aws\"] (close)"
2023-04-02T14:31:10.325-0700 [TRACE] dag/walk: vertex "root" is waiting for "provider[\"registry.terraform.io/hashicorp/aws\"] (close)"
2023-04-02T14:31:10.325-0700 [TRACE] dag/walk: vertex "provider[\"registry.terraform.io/hashicorp/aws\"] (close)" is waiting for "aws_elasticache_replication_group.redis (expand)"
2023-04-02T14:31:15.326-0700 [TRACE] dag/walk: vertex "root" is waiting for "provider[\"registry.terraform.io/hashicorp/aws\"] (close)"
2023-04-02T14:31:15.326-0700 [TRACE] dag/walk: vertex "provider[\"registry.terraform.io/hashicorp/aws\"] (close)" is waiting for "aws_elasticache_replication_group.redis (expand)"
2023-04-02T14:31:20.327-0700 [TRACE] dag/walk: vertex "root" is waiting for "provider[\"registry.terraform.io/hashicorp/aws\"] (close)"
2023-04-02T14:31:20.327-0700 [TRACE] dag/walk: vertex "provider[\"registry.terraform.io/hashicorp/aws\"] (close)" is waiting for "aws_elasticache_replication_group.redis (expand)"
2023-04-02T14:31:25.328-0700 [TRACE] dag/walk: vertex "root" is waiting for "provider[\"registry.terraform.io/hashicorp/aws\"] (close)"
2023-04-02T14:31:25.328-0700 [TRACE] dag/walk: vertex "provider[\"registry.terraform.io/hashicorp/aws\"] (close)" is waiting for "aws_elasticache_replication_group.redis (expand)"
2023-04-02T14:31:30.329-0700 [TRACE] dag/walk: vertex "provider[\"registry.terraform.io/hashicorp/aws\"] (close)" is waiting for "aws_elasticache_replication_group.redis (expand)"
2023-04-02T14:31:30.330-0700 [TRACE] dag/walk: vertex "root" is waiting for "provider[\"registry.terraform.io/hashicorp/aws\"] (close)"
2023-04-02T14:31:34.491-0700 [DEBUG] provider.terraform-provider-aws: HTTP Request Sent: @module=aws aws.operation=DescribeReplicationGroups

Pay attention to the timestamps.

Panic Output

No response

Important Factoids

No response

References

No response

Would you like to implement a fix?

Yes

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[jkoermer-eqxm]

Thank you for helping me understand why this is taking so long! I'm also seeing some long delays in my plans/applies with replication groups.

[stefansundin]

@jkoermer-eqxm Now we just have to wait 12+ months for HashiCorp to review my fix. 😭

[jkoermer-eqxm]

@stefansundin I spent all day today playing with the differences between terraform and the aws cli for encryption_in_transit. I was going to create a new issue for the requirement to recreate the cluster when enabling encryption in transit, but I see in your PR that you are resolving that issue as well. Is there another issue already for that? Or is it still worth creating one? I've added my upvote for your PR!

[stefansundin]

@jkoermer-eqxm I haven't seen an issue for that specifically but feel free to create one if you think that it might be useful. 👍

[mmarinaccio]

Just ran into this myself. Thanks for flagging. Hopefully we get a fix soon. Sucks having to needlessly add another 30s to our CI/CD workflows in order to manage this resource with Terraform.

[Leen15]

Is there any update on this?

Thanks

[github-actions]

Warning

This issue has been closed, meaning that any additional comments are hard for our team to see. Please assume that the maintainers will not see them.

Ongoing conversations amongst community members are welcome, however, the issue will be locked after 30 days. Moving conversations to another venue, such as the AWS Provider forum, is recommended. If you have additional concerns, please open a new issue, referencing this one where needed.

[github-actions]

This functionality has been released in v5.47.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!