Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Retry transient HTTP errors #972

Merged
merged 1 commit into from Dec 13, 2019
Merged

Retry transient HTTP errors #972

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
18 changes: 18 additions & 0 deletions docs/api-usage.rst
View file
Open in desktop
Expand Up @@ -348,3 +348,21 @@ throttled, you can set this parameter to -1. This parameter is ignored if
.. warning::

You will get an Exception, if you then go over the rate limit of your GitLab instance.

Transient errors
----------------

GitLab server can sometimes return a transient HTTP error.
python-gitlab can automatically retry in such case, when
``retry_transient_errors`` argument is set to ``True``. When enabled,
HTTP error codes 500 (Internal Server Error), 502 (502 Bad Gateway),
503 (Service Unavailable), and 504 (Gateway Timeout) are retried. By
default an exception is raised for these errors.

.. code-block:: python
import gitlab
import requests
gl = gitlab.gitlab(url, token, api_version=4)
gl.projects.list(all=True, retry_transient_errors=True)
6 changes: 5 additions & 1 deletion gitlab/__init__.py
View file
Open in desktop
Expand Up @@ -518,6 +518,8 @@ def http_request(

# obey the rate limit by default
obey_rate_limit = kwargs.get("obey_rate_limit", True)
# do not retry transient errors by default
retry_transient_errors = kwargs.get("retry_transient_errors", False)

# set max_retries to 10 by default, disable by setting it to -1
max_retries = kwargs.get("max_retries", 10)
Expand All @@ -531,7 +533,9 @@ def http_request(
if 200 <= result.status_code < 300:
return result

if 429 == result.status_code and obey_rate_limit:
if (429 == result.status_code and obey_rate_limit) or (
result.status_code in [500, 502, 503, 504] and retry_transient_errors
):
if max_retries == -1 or cur_retries < max_retries:
wait_time = 2 ** cur_retries * 0.1
if "Retry-After" in result.headers:
Expand Down