Backoff straitegy

[huan]

GRPC Connection Backoff Protocol

When we do a connection to a backend that fails, it is typically desirable to not retry immediately (to avoid flooding the network or the server with requests) and instead do some form of exponential backoff.

We have several parameters:

1. INITIAL_BACKOFF (how long to wait after the first failure before retrying)
2. MULTIPLIER (factor with which to multiply backoff after a failed retry)
3. JITTER (by how much to randomize backoffs).
4. MAX_BACKOFF (upper bound on backoff)
5. MIN_CONNECT_TIMEOUT (the minimum time we're willing to give a connection to complete)

gRPC Proposed Backoff Algorithm

ConnectWithBackoff()
  current_backoff = INITIAL_BACKOFF
  current_deadline = now() + INITIAL_BACKOFF
  while (TryConnect(Max(current_deadline, now() + MIN_CONNECT_TIMEOUT))
         != SUCCESS)
    SleepUntil(current_deadline)
    current_backoff = Min(current_backoff * MULTIPLIER, MAX_BACKOFF)
    current_deadline = now() + current_backoff +
      UniformRandom(-JITTER * current_backoff, JITTER * current_backoff)

See

• GRPC Connection Backoff Protocol