Should we add retry logic with exponential backoff to our API client?

[daoducanhh]

🏷️ Discussion Type

Question

Body

I've noticed that transient network errors and rate limits can cause API requests to fail, even when retrying the request a few seconds later would succeed.

Should we implement automatic retries in our API client using exponential backoff with jitter?

Guidelines

• I have read and understood this category's guidelines before making this post.

[dwcsnh]

I'd support this, with a few safeguards:

• Retry only transient failures. Handle 429 and selected 5xx responses, plus connection errors and timeouts. Don't retry most 4xx responses.

• Use capped exponential backoff with full jitter. For example, delay = random(0, min(maxDelay, baseDelay × 2^attempt)).

• Respect Retry-After. Avoid retrying earlier than the server-requested delay.

• Prevent duplicate side effects. Retry non-idempotent operations only when an idempotency key or equivalent mechanism makes them safe.

• Set an overall deadline. Three retries shouldn't cause a request to exceed the caller's timeout budget.