New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
XMLRPC is deprecated / ratelimit #22
Comments
I fixed it with this diff --git a/pip2arch.py b/pip2arch.py
index e0df1e5..d499c7d 100755
--- a/pip2arch.py
+++ b/pip2arch.py
@@ -7,6 +7,7 @@ import datetime
import logging
import argparse
import re
+import time
# make this script work for python2 and python3
# the try will fail on python3
@@ -15,6 +16,7 @@ try:
from xmlrpclib import ServerProxy
except NameError:
from xmlrpc.client import ServerProxy
+ import xmlrpc.client
BLANK_PKGBUILD = """\
#Automatically generated by pip2arch on {date}
@@ -48,6 +50,10 @@ class pip2archException(Exception): pass
class VersionNotFound(pip2archException): pass
class LackOfInformation(pip2archException): pass
+
+max_retries = 5
+retry_delay = 2 # seconds
+
class Package(object):
logging.info('Creating Server Proxy object')
client = ServerProxy('https://pypi.python.org/pypi')
@@ -67,10 +73,38 @@ class Package(object):
self.version = version
self.pyversion = pyversion
- data = self.client.release_data(name, version)
+ for retry in range(max_retries):
+ try:
+ data = self.client.release_data(name, version)
+ # Process the data as needed
+ break # Exit the loop if the request is successful
+ except xmlrpc.client.Fault as e:
+ if "HTTPTooManyRequests" in str(e):
+ print(f"Rate limit exceeded. Retrying in {retry_delay} seconds...")
+ time.sleep(retry_delay)
+ else:
+ print(f"An error occurred: {e}")
+ break # Exit the loop if another error occurs
+ else:
+ print("Max retries reached. Could not fetch data.")
+
logging.info('Got release_data from PyPi')
- raw_urls = self.client.release_urls(name, version)
+ for retry in range(max_retries):
+ try:
+ raw_urls = self.client.release_urls(name, version)
+ # Process the data as needed
+ break # Exit the loop if the request is successful
+ except xmlrpc.client.Fault as e:
+ if "HTTPTooManyRequests" in str(e):
+ print(f"Rate limit exceeded. Retrying in {retry_delay} seconds...")
+ time.sleep(retry_delay)
+ else:
+ print(f"An error occurred: {e}")
+ break # Exit the loop if another error occurs
+ else:
+ print("Max retries reached. Could not fetch data.")
+
logging.info('Got release_urls from PyPi')
if not len(data):
raise VersionNotFound('PyPi did not return any information for version {0}'.format(self.version)) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
They deprecated the xmlrpc and put a harsh rate limit on it:
https://warehouse.pypa.io/api-reference/xml-rpc.html
That's why it's now failing to do anything:
The text was updated successfully, but these errors were encountered: