Implement standard retry policies for failed API requests

[anikitin]

It seems that SDK supports one-time retry out of the box for failed requests. We need to support more sophisticated policies, especially for internal clients. I suggest to implement the following policy as a default retry policy:

• For HTTP 4xx errors (except HTTP 429) no retries are made
• For 429 and 5xx
  • Exponential backoff policy with jitter
  • Initial retry interval is configured: default = minimum = 1 second)
  • Number of retries is configured: minimum = 0, default = 3 retries, maximum should be limited, e.g. 20)
  • Jitter randomization factor is configured: minimum = 0 (no jitter), default = 0.3, maximum =0.5
  • If "Retry-After" header is returned by backend it has to be obeyed (current retry interval must NEVER be less than returned "Retry-After")

Client can define global policy or customize per API call.
SHould be eventually added to other SDKs as well.

See also:

• https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/
• https://www.baeldung.com/resilience4j-backoff-jitter

[tylerlong]

@embbnux Could you please add some comments about how integration team handles 429 issue in some of the popular production integration apps? We would like to take advantages of your experience. Thanks.

[embbnux]

The JS widget lib listen to 429 error, and show a message about the error to user with a time counter (from Retry-After) in UI. After time go to zero, it will request the API again.
https://github.com/ringcentral/ringcentral-js-widgets/blob/master/packages/ringcentral-integration/modules/RateLimiterV2/RateLimiter.ts#L113