How to use Polly retries or circuitbreaker for calling rate-limited APIs like Azure Cognitive Services

[danroot]

Some APIs throttle requests per second. In Azure Cognitive Services, the limit is 1/s for Basic, 10/s for Standard. Trello API similarly limits to 300/s. I would like to ensure that client code plays nice with those APIs even if incoming requests are greater than 1/s, and even if the client code is running on multiple machines. For example, if the client is in Azure Web App scaled out to multiple instances, and a burst of 1000 requests come in, I'd like to ensure that the client adheres to the 1/s, 10/s or 300/s rule as best as possible, while still using resources efficiently. #330 and #260 offer proposals for a rate-limit policy built in, which would be nice. However, is there some way to do similar with the current features in Polly? At least fail gracefully and back off for a few seconds on failure to avoid getting the API accounts banned.

[joelhulen]

@danroot You've inspired me to write a detailed blog post to help with this specific scenario: http://www.thepollyproject.org/2018/03/06/policy-recommendations-for-azure-cognitive-services/

Hopefully it can me more broadly applied to creating resiliency patterns for any rate-limiting service.

I'd really appreciate your feedback!

[danroot]

Very nice, thanks. There's karma to be had over on https://stackoverflow.com/questions/49031849/how-should-i-throttle-calls-from-my-c-sharp-client-in-azure-to-a-3rd-party-api-t if they reopen my question ;)