After switching to tenacity (#1234), we should start using exponential backoff instead of normal retries.