Introduce rate limit #1304
Comments
import requests
import time
### CustomField Translations - PLEASE ADJUST FOR YOUR CASE! ###################
story_points = "customfield_10006" # in WEB-UI it is written as cf[10006]
###############################################################################
### User-Parameter - PLEASE ADJUST FOR YOUR CASE! #############################
user = "VWUSERID"
token = "PleaseInsertYourRestAuthenticationApiTokenHere"
jql = "assignee = currentUser() and status = 'In Progress'"
return_fields = [
"summary", "reporter", "created", story_points
]
###############################################################################
### Instance and requests parameter set #######################################
max_results = 50
start_at = 0
rest_url = "https://my-domain/jira/rest/api/latest"
authentication = requests.auth.HTTPBasicAuth(user, token)
del user, token
###############################################################################
### REST query ################################################################
issue_counter = 0
request_counter = 1
sleep_time = 5
total_issues = 1
while issue_counter < total_issues:
rest_suffix = "search?jql={}&maxResults={}&startAt={}&fields={}".format(
jql,
max_results,
start_at,
",".join(return_fields)
)
request_url = rest_url + "/" + rest_suffix
response = requests.get(
request_url,
verify=True,
headers={"Content-Type": "application/json"},
auth=authentication
)
if response.status_code in [200, 201, 202]:
data = response.json()
issues = data['issues']
total_issues = data['total']
response_size = len(issues)
issue_counter += response_size
if response_size == 0:
continue
#### You can work with data here! #####################################
print("### {}. | {} issues ({}/{}) ###".format(
request_counter, response_size, issue_counter, total_issues
)
)
for issue in issues:
print(
"- Key: '{}', summary: '{}', created: '{}', story points: '{}'".format(
issue['key'], issue['fields']['summary'],
issue['fields']['created'], issue['fields'][story_points]
)
)
#######################################################################
start_at += max_results
request_counter += 1
elif response.status_code == 429:
print("Rate limit exceeded! Wait {}s and redo.".format(sleep_time))
time.sleep(sleep_time)
else:
print(response.content)
break
sleep_time = float(response.headers['Retry-After'])*1.2
if sleep_time > 0:
print("Rate limit reached! Wait {}s.".format(sleep_time))
time.sleep(sleep_time)
else:
print("\nDone.")This is a python script our support provided me that makes use of REST API rate limit handling (2nd recommendation from Atlassian), field selection, chunked issue query (Jira) and token. Maybe it helps implementing a solution |
|
Thanks for reporting this, while i am working on #1292 i think i will in the same enhance the ConfluenceClient and add a rate limit feature as this is for now the simplest and most configurable solution. |
|
Added and released in v3.2.0 :) |
|
Unfortunately, it is not working with version 3.2.0 at all. 3.1.2 is still working without error 400. It doesn't matter if I set the new option |
|
Can you please append |
That is correct. The stacktrace of |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is your feature request related to a problem? Please describe.
I use the task
publishToconfluenceon a self-hosted and managend Confluence where I run into 429 status code. After some investigations I figured out that the Confluence API on our system is protected by a rate limit 1 requests per second (1 req/sec). Therefore, it is not possible for me to upload our entire documentation to confluence at the moment.Describe the solution you'd like
It would be nice to have an option like
Atlassian provides possible solutions: https://confluence.atlassian.com/adminjiraserver/adjusting-your-code-for-rate-limiting-987143384.html
The solution "1. Exponential backoff" sounds good to me.
Additional context
Unfortunately, I don't feel confident to provide a PR.
The text was updated successfully, but these errors were encountered: