Skip to content

Poller autoscaler: add backoff on ResourceExhausted errors#1110

Merged
mjameswh merged 4 commits intomasterfrom
autoscaller-backoff-on-exhausted
Feb 12, 2026
Merged

Poller autoscaler: add backoff on ResourceExhausted errors#1110
mjameswh merged 4 commits intomasterfrom
autoscaller-backoff-on-exhausted

Conversation

@mjameswh
Copy link
Copy Markdown
Contributor

What was changed

  • Add exponential backoff for ResourceExhausted in autoscaling pollers

Why?

Previously, when autoscaling pollers would hit ResourceExhausted errors (e.g., namespace rate limits), they were retrying immediately with no backoff, creating a sustained hot loop that could generate at extremely high rate (over 60Kreq/seq in local simulations, but more realistically a few hundreds assuming normal network capacity).

This PR adds exponential backoff logic to the poller autoscaller, with the same configuration as is applied by default by the client on ResourceExhausted errors, i.e.: (1s initial, 10s max, 2x multiplier).

How was it tested?

  • Added unit test
  • Ran e2e tests locally using the Python SDK

@mjameswh mjameswh requested a review from a team as a code owner February 12, 2026 11:58
Copy link
Copy Markdown
Member

@chris-olszewski chris-olszewski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems sensible to me

Comment thread crates/sdk-core/src/pollers/poll_buffer.rs Outdated
Copy link
Copy Markdown
Member

@Sushisource Sushisource left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thanks James, just one small suggestion.

Comment thread crates/sdk-core/src/pollers/poll_buffer.rs Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants