fix: Add retries to SAR service calls on throttle #2352
Conversation
| from enum import Enum | ||
|
|
||
| LOG = logging.getLogger(__name__) | ||
|
|
||
|
|
||
| class SamPlugins(object): | ||
| """ |
There was a problem hiding this comment.
Move this to a separate file to resolve a circular dependency.
samtranslator.plugins (__init__.py) -> samtranslator.model.exceptions… -> samtranslator.plugins.LifeCycleEvents
Codecov Report
@@ Coverage Diff @@
## develop #2352 +/- ##
===========================================
+ Coverage 93.58% 94.31% +0.73%
===========================================
Files 90 98 +8
Lines 6124 7127 +1003
Branches 1260 1468 +208
===========================================
+ Hits 5731 6722 +991
- Misses 183 196 +13
+ Partials 210 209 -1
Continue to review full report at Codecov.
|
| def _make_service_call_with_retry(self, service_call, app_id, semver, key, logical_id): | ||
| call_succeeded = False | ||
| start_time = time() | ||
| while (time() - start_time) < self.TEMPLATE_WAIT_TIMEOUT_SECONDS: |
There was a problem hiding this comment.
There's also a similar retry pattern used in on_after_transform_template. I can see things are done a bit differently within this retry and that retry. But can we try to refactor the retry part into a decorator that can be used on both?
There was a problem hiding this comment.
both on_after_transform_template && on_before_transform_template functions at the end they call _sar_service_call to do the SAR service call, so I think we can add our retry logic in _sar_service_call call.
But we need to make sure that the waiting time we will use in retry is cumulative between all retials in the different paths, and this cumulative value should be less than TEMPLATE_WAIT_TIMEOUT_SECONDS
Issue #, if available:
Description of changes:
Description of how you validated changes:
Checklist:
make prpassesExamples?
Please reach out in the comments, if you want to add an example. Examples will be
added to
sam initthrough https://github.com/awslabs/aws-sam-cli-app-templates/By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.