-
Notifications
You must be signed in to change notification settings - Fork 75
feat(go-sdk): add rate limiter with exponential backoff and circuit breaker #161
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
VedantMadane
wants to merge
6
commits into
Agent-Field:main
Choose a base branch
from
VedantMadane:feat/go-sdk-rate-limiter
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
feat(go-sdk): add rate limiter with exponential backoff and circuit breaker #161
VedantMadane
wants to merge
6
commits into
Agent-Field:main
from
VedantMadane:feat/go-sdk-rate-limiter
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…reaker Adds production-grade rate limiting to the Go SDK AI client with: - Exponential backoff for retry logic - Jitter to prevent thundering herd - Circuit breaker pattern (Closed/Open/HalfOpen states) - Automatic rate limit error detection - Support for both regular and streaming AI calls Configuration options added to ai.Config: - RateLimitMaxRetries (default: 5) - RateLimitBaseDelay (default: 1s) - RateLimitMaxDelay (default: 30s) - RateLimitJitterFactor (default: 0.1) - CircuitBreakerThreshold (default: 5) - CircuitBreakerTimeout (default: 60s) - DisableRateLimiter (default: false) Includes comprehensive test coverage (18 new tests) and documentation. Fixes Agent-Field#97
…er, observability Implements all critical and major fixes from code review: **CRITICAL FIXES:** - Add mutex protection for thread-safe concurrent access - Fix broken documentation link (removed reference to deleted file) - Add sync.Mutex to RateLimiter struct to protect mutable state - Lock access to consecutiveFailures and circuitOpenTime fields **MAJOR FIXES:** - Fix jitter implementation to use time-based randomness * Was deterministic per container, defeating thundering herd prevention * Now uses time.Now().UnixNano() for true randomness - Move rate limiter defaults to DefaultConfig() * Provides consistent API - defaults visible before client creation * Simplifies NewClient() logic - Add circuit breaker observability * GetCircuitState() - check current circuit state * GetConsecutiveFailures() - monitor failure count * OnCircuitOpen/OnCircuitClose callbacks for metrics/logging **TEST IMPROVEMENTS:** - Add TestRateLimiter_ConcurrentAccess (100 concurrent goroutines) - Add streaming retry tests: * TestExecuteStreamWithRetry_Success * TestExecuteStreamWithRetry_RateLimitThenSuccess * TestExecuteStreamWithRetry_MaxRetriesExceeded * TestExecuteStreamWithRetry_NonRateLimitError - Add TestGetCircuitState for observability methods - Add TestCircuitBreakerCallbacks for callback verification **DOCUMENTATION:** - Fix README.md broken link to RATE_LIMITER_USAGE.md - Add inline Rate Limiting section with configuration examples - Document thread safety guarantees - Add production monitoring examples Thread Safety: RateLimiter is now safe for concurrent use. Breaking Changes: None - all changes are backward compatible. Test Coverage: Added 9 new tests (total: 27 tests) Addresses PR Agent-Field#161 code review feedback.
Contributor
|
@VedantMadane will review but youll need to sign the CLA before we can merge |
Contributor
|
tests failing |
Author
|
Could you approve the GitHub Actions workflow..? |
Contributor
Performance
✓ No regressions detected |
Contributor
|
bumping, looks like still failing |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Adds production-grade rate limiting to the Go SDK AI client to match functionality in Python and TypeScript SDKs. Implements exponential backoff with jitter and circuit breaker pattern for resilient AI API calls.
Changes
New Files
sdk/go/ai/rate_limiter.go- Rate limiter implementation (373 lines)sdk/go/ai/rate_limiter_test.go- Comprehensive test suite (18 tests, all passing)sdk/go/ai/RATE_LIMITER_USAGE.md- Usage documentation and examplesModified Files
sdk/go/ai/config.go- Added rate limiter configuration optionssdk/go/ai/client.go- Integrated rate limiter into Complete() and StreamComplete() methodssdk/go/ai/README.md- Added rate limiter feature documentationFeatures
✅ Exponential Backoff: Delays increase exponentially (1s → 2s → 4s → 8s...)
✅ Jitter: Container-specific randomization prevents thundering herd
✅ Circuit Breaker: Opens after N consecutive failures, prevents cascade
✅ Automatic Detection: Identifies rate limit errors from status codes and error messages
✅ Configurable: All parameters tuneable via ai.Config
✅ Opt-out: Can be disabled with DisableRateLimiter flag
Developer Experience
Testing
All tests pass (176 total, 18 new):
go test ./... -vNew test coverage includes:
Reference Implementation
Based on Python SDK
rate_limiter.pywith Go-idiomatic patterns:Compatibility
Documentation
See RATE_LIMITER_USAGE.md for:
Fixes #97