plugin/route53: make refresh frequency adjustable #3083
Conversation
|
Thank you for your contribution. I've just checked the OWNERS files to find a suitable reviewer. This search was successful and I've asked yongtang (via If you have questions or suggestions for this bot, please file an issue against the miekg/dreck repository. The bot understands the commands that are listed here. |
|
there seems to be many issue to the frequency that allows you to fetch stuff from this API. A more user friendly approach is to figure out programmatically what the best freq. is and use that. |
|
The Terraform approach is to make as many requests as you can and then use an exponential back-off until you've made all the requests you want. The problem there is that you don't know if you're going to be limited until you are and once you are, you can only wait and keep trying. That works fine from the standpoint of a single client, but the rate-limits are global to your whole AWS account. So too many clients without knowledge of each other trying to pull as many records as fast as possible can quickly degrade your AWS API service quality. As an operator, I have the view of how many services I have running that need to interface with it and I need to be able to control them to prevent rate-limits from being reached in the first place to the best of my ability. This isn't a huge deal when I'm running one CoreDNS daemon configured to pull from Route 53, but it increasingly becomes an issue for every additional replica added that is configured this way. |
Codecov Report
@@ Coverage Diff @@
## master #3083 +/- ##
==========================================
+ Coverage 55.85% 55.92% +0.06%
==========================================
Files 205 205
Lines 10108 10123 +15
==========================================
+ Hits 5646 5661 +15
Misses 4051 4051
Partials 411 411
Continue to review full report at Codecov.
|
the current update frequency for the refresh loop in the route 53 plugin is hard-coded to 1 minute. aws rate-limits the number of api requests so less frequent record refreshes can help when reaching those limits depending upon your individual scenarios. this pull adds a configuration option to the route53 plugin to adjust the refresh frequency. thanks for getting my last pull released so quickly. this is the last local change that i have been running and would love to get it contributed back to the project. Signed-off-by: Matt Kulka <mkulka@parchment.com>
1. Why is this pull request needed and what does it do?
the current update frequency for the refresh loop in the route 53 plugin is hard-coded
to 1 minute. aws rate-limits the number of api requests so less frequent record refreshes
can help when reaching those limits depending upon your individual scenarios. this pull
adds a configuration option to the route53 plugin to adjust the refresh frequency.
thanks for getting my last pull released so quickly. this is the last local change that
i have been running and would love to get it contributed back to the project.
2. Which issues (if any) are related?
Another step to solving #2353
3. Which documentation changes (if any) need to be made?
Included in pull.
4. Does this introduce a backward incompatible change or deprecation?
No, maintains current behavior by default.