Skip to content
This repository
Browse code

Also scrapes https pages.

  • Loading branch information...
commit 13feb89274bc8a21e40518a25b6ecadb7f4a9b53 1 parent 64d0685
Edho Arief authored June 13, 2012 spladug committed June 26, 2012

Showing 1 changed file with 1 addition and 1 deletion. Show diff stats Hide diff stats

  1. 2  r2/r2/lib/scraper.py
2  r2/r2/lib/scraper.py
@@ -96,7 +96,7 @@ def fetch_url(url, referer = None, retries = 1, dimension = False):
96 96
     nothing = None if dimension else (None, None)
97 97
     url = clean_url(url)
98 98
     #just basic urls
99  
-    if not url.startswith('http://'):
  99
+    if not (url.startswith('http://') or url.startswith('https://')):
100 100
         return nothing
101 101
     while True:
102 102
         try:

0 notes on commit 13feb89

Please sign in to comment.
Something went wrong with that request. Please try again.