Add repeat requests helper method for Google APIs (GSuite) #169
Conversation
skiptomyliu
commented
Oct 1, 2019
•
skiptomyliu
commented
- We make a lot of requests to GSuite. This makes it more resilient in case of a Google server blip, which we've seen.
- Add retry for GCP compute fetching.
- Refactor the transform objects method to be more general to prevent copy-pasta.
skiptomyliu
changed the title
skiptomyliu
force-pushed
the
gsuite-retry2
branch
from
4d33807
to
77a5aa0
Compare
| logger = logging.getLogger(__name__) | ||
|
|
||
|
|
||
| def repeat_request(req, req_args, req_next=None, retries=5, retry_delay_ms=500): |
There was a problem hiding this comment.
Could we implement this as generic retry operation we can use across all modules and tasks?
For example of method signature:
invoke_with_retry(operation, retries, retry_delay, retriable_exception_types[])
skiptomyliu
changed the title
|
As @sachafaust, @ecdavis, and I discussed earlier, let's make sure that the cleanup stage does not get executed if all of the retries fail. |
skiptomyliu
changed the title
|
#184 seems to handle the GSuite work. I wonder if we could use a similar approach for GCP in this PR. |
Just updated GCP with the retries. |
skiptomyliu
force-pushed
the
gsuite-retry2
branch
from
54288ad
to
bfe555e
Compare
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. |
|
This issue has been automatically closed for inactivity. If you still wish to make these changes, please open a new change or reopen this one. |
