Skip to content
Browse files

Modify HTTP Error behavior to act as previously; Implemeted Retry Cou…

…nter to avoid infinite loop
  • Loading branch information...
1 parent d879f42 commit f6671ba0a5c1841c977f538a241f2e089e8fee7d spikepmw committed Feb 12, 2011
Showing with 14 additions and 5 deletions.
  1. BIN Compiled Binary/MAC/manga
  2. +14 −5 helper.py
View
BIN Compiled Binary/MAC/manga
Binary file not shown.
View
19 helper.py
@@ -60,15 +60,15 @@ def fixFormatting(s):
# version = 'Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.472.14 Safari/534.3'
-def getSourceCode(url):
+def getSourceCode(url, maxRetries=5, waitRetryTime=5):
# overwrite default user-agent so we can download
UserAgent = 'Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.472.14 Safari/534.3'
"""
While loop to get around server denies for info or minor disconnects.
"""
- ret = None
while True:
+ ret = None
try:
# Modifies the user agent and notifies the server we accept gzip encoding
request = urllib2.Request(url)
@@ -89,9 +89,18 @@ def getSourceCode(url):
ret = gzipper.read()
else:
raise FatalError('Uknown HTTP Encoding returned')
-
- except IOError:
- pass
+ except urllib2.URLError, e:
+ if e.code == 404:
+ ret = ""
+ break
+ else:
+ # Check maxrRetries. If we should try again, wait waitRetryTime seconds
+ if (maxRetries > 0):
+ time.sleep(waitRetryTime)
+ maxRetries = maxRetries - 1
+ else:
+ # We have already tries maxRetries times. Just break to avoid an infinite loop
+ break
else:
break

0 comments on commit f6671ba

Please sign in to comment.
Something went wrong with that request. Please try again.