Rate limit with wait and retry of the remaining piece of the time window #930
Comments
|
The As to your original question my (untried and untested) suggestion is to:
It might be that there's some context missing somewhere that means that doesn't quite work, but it might give you something to go on to get it to do what you want. It's possible that this wasn't a use case catered for in the original design, and it isn't currently possible with the functionality available via the public API of Polly. |
|
@martincostello this is the complete working example I used to simulate 40 API calls in parallel, with a limit of 3 calls every 5 seconds: int maxCalls = 3;
int intervalInSeconds = 5;
var rateLimitPolicy =
Policy
.RateLimitAsync(maxCalls, TimeSpan.FromSeconds(intervalInSeconds), maxCalls);
var retryPolicy =
Policy
.Handle<RateLimitRejectedException>()
.WaitAndRetryAsync
(
retryCount: 3,
retryNumber => TimeSpan.FromSeconds(intervalInSeconds)
)
.WrapAsync(rateLimitPolicy);
ConcurrentBag<Task> tasks = new ();
Parallel
.ForEach
(
Enumerable.Range(1, 40),
n =>
{
Task t =
retryPolicy
.ExecuteAsync(() =>
{
Console.WriteLine($"{DateTime.Now:HH:mm:ss:ff} - Call #{n}");
return Task.CompletedTask;
});
tasks.Add(t);
}
);
await Task.WhenAll(tasks);Now if I try to use the public static AsyncRateLimitPolicy<TResult> RateLimitAsync<TResult>(int numberOfExecutions, TimeSpan perTimeSpan, Func<TimeSpan, Context, TResult> retryAfterFactory)If I try to implement |
|
My mistake, the delegate has a different purpose with rate limiting. The delegate is used to return the result to return if the execution is rate limited. The Polly/src/Polly/RateLimit/RateLimitEngine.cs Lines 23 to 28 in cd3d697 Polly/src/Polly/RateLimit/AsyncRateLimitEngine.cs Lines 25 to 30 in cd3d697 On that basis you should be able to get what you're trying to achieve like this: [Fact]
public async Task Rate_Limit_Wait_Until_Retry_After()
{
int maxCalls = 3;
int intervalInSeconds = 5;
var rateLimitPolicy =
Policy
.RateLimitAsync(maxCalls, TimeSpan.FromSeconds(intervalInSeconds), maxCalls);
var retryPolicy =
Policy
.Handle<RateLimitRejectedException>()
.WaitAndRetryAsync(3, (retryNumber, context) => (TimeSpan)context["Retry-After"], // This will have a value of 42 seconds
(_, _, _) => { })
.WrapAsync(rateLimitPolicy);
await retryPolicy.ExecuteAsync(context =>
{
try
{
// Your API call goes here - throwing the exception is just for demonstration purposes
throw new RateLimitRejectedException(TimeSpan.FromSeconds(42));
}
catch (RateLimitRejectedException ex)
{
context.Add("Retry-After", ex.RetryAfter);
throw;
}
},
new Context());
} |
|
@martincostello thank you for your help. |
|
Yes, you need to add it to your code as-per the example. |
|
@martincostello how can I throw the |
|
Just like in the code sample:
|
|
Sorry, I just realised there was something I forgot to do in the original code sample, I'll see if I can extend it. Ultimately though, you need to layer the policies to achieve what you want - you can't do it out of the box by just wrapping one policy in another. |
|
@martincostello you're absolutely right, since the retry needs to work on the single call while the rate-limit must be defined globally (for all calls). class Program
{
static async Task Main(string[] args)
{
ClientWithPolicy client = new(); //Dependency injection
Task thread1 =
Task.Run(async () =>
{
while (true)
{
Random r = new();
int random = r.Next();
if (random % 2 == 0)
{
await client.DoCallWithPolicyAsync(1);
}
if (random % 5 == 0)
{
Console.WriteLine($"Thread #1 - Delaying 6 seconds");
await Task.Delay(6000);
}
}
});
Task thread2 =
Task.Run(async () =>
{
while (true)
{
Random r = new();
int random = r.Next();
if (random % 2 != 0)
{
await client.DoCallWithPolicyAsync(2);
}
if (random % 5 == 0)
{
Console.WriteLine($"Thread #2 - Delaying 8 seconds");
await Task.Delay(8000);
}
}
});
await Task.WhenAll(thread1, thread2);
}
}
class ClientWithPolicy
{
private int _count = 0;
private readonly AsyncPolicy _policy = Policy.RateLimitAsync(5, TimeSpan.FromSeconds(5), 5);
public async Task DoCallWithPolicyAsync(int threadNumber)
{
int c = Interlocked.Increment(ref _count);
var retryPolicy =
Policy
.Handle<RateLimitRejectedException>()
.WaitAndRetryAsync
(
retryCount: 3,
retryNumber => TimeSpan.FromMilliseconds(100),
(ex, ts, n, ctx) => Console.WriteLine($"Thread #{threadNumber} - Call #{c} - retry attempt #{n}")
);
await retryPolicy
.ExecuteAsync(async () =>
{
try
{
await _policy.ExecuteAsync(() => DoCallAsync(threadNumber, c));
}
catch (RateLimitRejectedException ex)
{
await Task.Delay(ex.RetryAfter);
throw;
}
});
}
private static Task DoCallAsync(int threadNumber, int callNumber)
{
Console.WriteLine($"{DateTime.Now:HH:mm:ss:ff} Thread #{threadNumber} - Doing the call #{callNumber}");
return Task.CompletedTask;
}
}Producing an output: |
Summary: What are you wanting to achieve?
I have a rate-limited API that I need to call, limits are of 300 calls for 5 minutes.
I wanted to implement a rate limit with a retry and I did successfully with this code:
Here I'm using the Polly.Contrib.WaitAndRetry package to help with the WaitAndRetry policy.
With this code I'm able to do what I want, but once the limit is reached the code will wait for the entire time window (5 mins) before trying again. I'd like to wait only for the remaining part of the time window, this info is inside the exception
RateLimitRejectedExceptionbut I have no idea how to set it as the wait time in the retry policy.Also, I read in docs of a factory
retryAfterFactorybut I don't understand how to use it, since the policy becomes typed onTimeSpanand when callingExecutewants a TimeSpan back:How can I achieve this behavior?
The text was updated successfully, but these errors were encountered: