[ENHANCEMENT] Honor provider Retry-After on 429 in embedder (faster indexing)

[hannesrudolph]

Type

Enhancement

Problem / Value

When embedding requests hit provider rate limits (e.g., 429), providers like Gemini and others return a specific cooldown (Retry-After or similar). The current behavior uses fixed/exponential delays and a global wait, which can either slow down users who aren’t hitting limits or keep retrying too soon and waste quota. Respecting the provider’s exact retry guidance improves throughput and avoids unnecessary slowdowns or quota exhaustion.

Context

Affects code indexing that generates embeddings across providers (including Gemini via the OpenAI-compatible endpoint). Providers often include a clear retry window or structured rate-limit info in the response. Today, retries use guessed backoff and a shared global delay rather than the provider’s suggested wait. Users indexing larger projects are most impacted; honoring provider signals would keep indexing fast for those not rate-limited and recover efficiently when limits are hit.

Constraints/Preferences

• Parse and use the provider’s Retry-After or equivalent backoff signal on 429 responses.
• For Gemini, also parse structured retry info when present.
• Only delay when a rate limit is actually hit; do not introduce per-request delays otherwise.
• Fall back to exponential backoff only if no provider guidance is available.
• Keep a reasonable max cap and avoid log flooding; retain global coordination but base it on provider-provided reset times.

[roomote]

I'm exploring a potential fix for this enhancement to honor provider-specific retry guidance on 429 responses. I'll open a PR to share progress.

[roomote]

I've opened PR #8105 with my implementation to address this enhancement. The solution:

✅ Parses and honors Retry-After headers from providers (supports multiple formats)
✅ Handles Gemini's structured retry info in error responses
✅ Falls back to exponential backoff only when no provider guidance is available
✅ Maintains a reasonable max cap (5 minutes) and global coordination
✅ Includes comprehensive test coverage

The implementation should improve embedding throughput by respecting provider-specific retry windows, reducing unnecessary delays and quota exhaustion.

PR: #8105