Support additional retry codes

[mbamber]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

The AWS SDK v2 supports retrying additional error codes with retry.AddWithErrorCodes(). It would be useful to be able to add specific error codes that can be retried.

Specifically we are interested in ensuring that an AccessDeniedException is retried as our credential vending process may not have all the required permissions at the start of the Terraform run (the specific reasons why we need this approach are complicated and beyond the scope of this ticket, although if this context would help then please feel free to reach out to me).

I would not expect many people to make use of this functionality (as Terraform already does an excellent job of retrying when needed) but I believe that this change can be fully backwards compatible and transparent if needed.

Potential Library Implementation

// sdk v2
if retryCodes := os.Getenv("AWS_RETRY_CODES"); retryCodes != "" {
  codes := strings.Split(retryCodes, ",")
  retryer = retry.AddWithErrorCodes(retryer, codes...)
}

// sdk v1 shim
if retryCodes := os.Getenv("AWS_RETRY_CODES"); retryCodes != "" {
  codes := strings.Split(retryCodes, ",")
  log.Printf("[DEBUG] Using additional retry codes: %s", codes)
  sess.Handlers.Retry.PushBack(func(r *request.Request) {
    if tfawserr.ErrCodeEquals(r.Error, codes...) {
      r.Retryable = aws.Bool(true)
    }
  })
}

Potential Terraform Backend/Provider Configuration

This could be exposed via a provider level setting, or simply just as an environment variable such as AWS_RETRY_CODES.

[mbamber]

I've created #295 which closes this. Any help greatly appreciated

[mbamber]

Hi @gdavison - is this something that you'd consider merging into the upstream? We'd like to have an idea of how likely this would be to be merged before we continue with our internal projects in this area

[gdavison]

Hi @mbamber. We're doing some thinking on how we currently handle retries right now. We'll add this to what we're considering