Skip to content

Conversation

@JBWilkie
Copy link
Contributor

@JBWilkie JBWilkie commented Feb 1, 2025

Problem

Darwin recently introduced some API-wide rate limits. If they are exceeded, they will return an HTTP 429 error. Historically, darwin-py has not had handling for 429 errors, except for 1 specific functionality: Registering items from external storage

Solution

Resolve this by:

  • 1: Applying the existing backoff and retry code (implemented with tenacity) to the generic get, post, put, and delete Client class methods
  • 2: Funnel as many Darwin API calls through these methods as is practical and reasonable

We also include the 500, 502, 503 and 504 as codes that trigger retries - These were previously used to retry some requests in darwin-py as protection against temporary drops in external API services

There are some exceptions that are acceptable - Such as in client.from_api_key() when creating a Client object. This is because the back & retry code is applied to Client methods

Changelog

Introduced API-wide rate limit and server error retry handling intro darwin-py

@linear
Copy link

linear bot commented Feb 1, 2025

@JBWilkie JBWilkie force-pushed the DAR-5422 branch 2 times, most recently from c0d21a5 to 1605898 Compare February 1, 2025 14:42
@JBWilkie JBWilkie merged commit ef13841 into master Feb 5, 2025
20 checks passed
@JBWilkie JBWilkie changed the title [DAR-5422][External] API-wide HTTP 429 handling [DAR-5422][External] API-wide HTTP 429 and 500 handling Feb 5, 2025
@JBWilkie JBWilkie changed the title [DAR-5422][External] API-wide HTTP 429 and 500 handling [DAR-5422][External] API-wide HTTP 429 and 5xx handling Feb 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants