add rate limit handling #25327
add rate limit handling #25327
Conversation
content-bot
changed the base branch from
master
to
contrib/jthom-vmray_add-rate-limit-handling
|
Thank you for your contribution. Your generosity and caring are unrivaled! Make sure to register your contribution by filling the Contribution Registration form, so our content wizard @daryakoval will know the proposed changes are ready to bereviewed. |
content-bot
added
Contribution Form Filled
…ate-limit-handling
| demisto.debug(f"Rate limit exceeded! Waiting {seconds_to_wait} seconds and then retry.") | ||
| time.sleep(seconds_to_wait) # pylint: disable=sleep-exists | ||
| reset_file_buffer(files) | ||
| return http_request(method, url_suffix, params, files, ignore_errors, get_raw) |
There was a problem hiding this comment.
In this case, it would be better to assign the response to a variable and then continue with the logic of the function.
| return http_request(method, url_suffix, params, files, ignore_errors, get_raw) | |
| r = http_request(...) |
Why are the request args different from the one above?
| return http_request(method, url_suffix, params, files, ignore_errors, get_raw) | |
| r = requests.request( | |
| method, url, params=params, headers=HEADERS, files=files, verify=USE_SSL, proxies=PROXIES | |
| ) |
There was a problem hiding this comment.
The arguments differ, because it is a recursive function call and not a direct request. We need to call http_request recursively here, since it is possible that retrying only once could end up in a rate limit that was triggered by another waiting request.
There was a problem hiding this comment.
Missed the recursion. So what is the stop condition? How do you make sure you won't enter an infinite loop and get a stack overflow? you should count the number of tries.
Co-authored-by: Dan Sterenson <38375556+dansterenson@users.noreply.github.com>
Co-authored-by: Dan Sterenson <38375556+dansterenson@users.noreply.github.com>
Co-authored-by: Dan Sterenson <38375556+dansterenson@users.noreply.github.com>
Co-authored-by: Dan Sterenson <38375556+dansterenson@users.noreply.github.com>
| demisto.debug(f"Rate limit exceeded! Waiting {seconds_to_wait} seconds and then retry.") | ||
| time.sleep(seconds_to_wait) # pylint: disable=sleep-exists | ||
| reset_file_buffer(files) | ||
| return http_request(method, url_suffix, params, files, ignore_errors, get_raw) |
There was a problem hiding this comment.
Missed the recursion. So what is the stop condition? How do you make sure you won't enter an infinite loop and get a stack overflow? you should count the number of tries.
dansterenson
merged commit 41d53ec
into
demisto:contrib/jthom-vmray_add-rate-limit-handling
* add rate limit handling * Update Packs/VMRay/Integrations/VMRay/README.md * Update Packs/VMRay/Integrations/VMRay/VMRay.py * Update Packs/VMRay/Integrations/VMRay/VMRay.py * Update Packs/VMRay/ReleaseNotes/1_1_7.md * use 'call_count' to verify rate limit triggered * add counter for rate limit reties * add unit test for max reties --------- Co-authored-by: Jens Thom <72789379+jthom-vmray@users.noreply.github.com> Co-authored-by: Dan Sterenson <38375556+dansterenson@users.noreply.github.com>
Contributing to Cortex XSOAR Content
Make sure to register your contribution by filling the contribution registration form
The Pull Request will be reviewed only after the contribution registration form is filled.
Status
Related Issues
fixes: link to the issueDescription
The app could not handle rate limiting, which lead to failing playbook runs. This PR introduces a new setting that handles rate limiting, if set.
Screenshots
Paste here any images that will help the reviewerMinimum version of Cortex XSOAR
Does it break backward compatibility?
Must have