Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Added request cache and queue to AWS provider and use it from variable resolution #4518
What did you implement:
How did you implement it:
Added an on-demand request cache to the AWS provider. The cache can be utilized and activated by adding a new last parameter to the
The requests intiated from the variable system will set the cached flag to let the variable resolution request static info like the stack descriptions only once.
This PR complements #4499 which added the generic cache for the variable system. Together the fixes will eliminate the rate exceeded errors.
How can we verify it:
Add lots of external variable references (e.g.
Is this ready for review?: YES
changed the title from
Added request cache to AWS provider and use it from variable resolution [WIP]
Added request cache to AWS provider and use it from variable resolution
Dec 4, 2017
@e-e-e I changed the cache to the same semantics as used for the variable cache now (implicit promise queue). Now it should be safe against excessive request calls. Please double check.
I will add an additional queue, as requests should be done in parallel with bluebird queue.
@e-e-e Now it uses a queue for all requests sent (and restricts the maximum parallel requests to 2 at a time) in combination with the result caching. So subsequent duplicate requests are cached on-demand, and all different requests eventually submitted are queued.
Additionally did a dependency security check (to test for the new dependencies). No security issues have been found:
referenced this pull request
Dec 5, 2017
Thank you for implementing @HyperBrain and following up @e-e-e
How did you guys use the yml file looks like?
Here is the yml file which the rate exceeded error occurs.