Cache existing task definitions to avoid rate limiting #4177
Conversation
ldez
added
status/1-needs-design-review
kind/enhancement
emtammaru
left a comment
emtammaru
left a comment
There was a problem hiding this comment.
We are having the exact same issue with the rate-limiting due to multiple traefik instances in different environments using the same AWS account with shared API limits. This would help us tremendously -- was planning to do the same exact optimization here.
ghost
left a comment
ghost
left a comment
There was a problem hiding this comment.
This is really needed and would help tremendously.
|
We have grown our usage of Traefik within AWS and it appears that we have hit this limitation as well.\ with several environments in an account each with several traefik instances we are hitting the API rate limiting constantly. The logic in the original implementation abuses the AWS describe task api at any reasonable scale. This request addresses much of it by reducing duplication. Might also want a delay/back off when receiving 429 for describe task. |
ldez
added
status/2-needs-review
and removed
contributor/waiting-for-corrections
status/1-needs-design-review
labels
ldez
added
kind/bug/fix
ldez
changed the title
7 participants
What does this PR do?
Stores existing task definitions to avoid rate limiting of the AWS describeTask API. If there is an existing definition, use instead of making another API call.
Motivation
Deployments taking hours instead of minutes. Due to using multiple instances of Traefik, each making frequent desribeTask API calls, and quickly hitting AWS's rate limit.
More
Additional Notes