Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introducing RateLimiter to Control Request Rate (Issue #4) #5

Merged
merged 4 commits into from
Jun 23, 2023

Conversation

0x19
Copy link
Contributor

@0x19 0x19 commented Jun 21, 2023

Summary:

In this Pull Request, we have introduced a RateLimiter struct in the sourcify package to control the rate at which HTTP requests are sent from the client to the server. The rate limiter is based on the token bucket algorithm and provides a mechanism to prevent an HTTP client from exceeding a specified rate of requests.

Key points in the PR:

  • Added a new RateLimiter struct that maintains a token bucket. Tokens are added to the bucket at a specified rate, limiting the rate at which requests can be sent. If a request is made when the bucket is empty, the request is blocked until a token is available, ensuring the rate of requests does not exceed the specified rate.
  • The RateLimiter struct includes a Wait() method, which is used to control the rate of requests. Before sending a request, the client calls the Wait() method to consume a token from the bucket.
  • Integrated the RateLimiter into the Client struct. The Client now includes a RateLimiter that is used to control the rate of requests when the doRequestWithRetry method is called.
  • Updated existing unit tests to include rate limiting and added new unit tests specifically for the RateLimiter.

By introducing the RateLimiter, we aim to improve the reliability and robustness of the client by preventing it from overwhelming the server with requests and being rate-limited or banned. This also promotes more efficient use of server resources.

WARN: Prior this PR is merged, #6 needs to be resolved first.

This PR closes #4

@0x19 0x19 self-assigned this Jun 21, 2023
@0x19 0x19 linked an issue Jun 21, 2023 that may be closed by this pull request
@0x19 0x19 merged commit 74f9e3a into main Jun 23, 2023
8 checks passed
@0x19 0x19 deleted the client/rate_limit branch June 23, 2023 14:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add client rate limit support
1 participant