Implemented support for basic authentication #38
Conversation
| @@ -50,7 +51,8 @@ class Scraper(object): | |||
| """Generic class to download an application from the Mozilla server""" | |||
|
|
|||
| def __init__(self, directory, version, platform=None, | |||
| application='firefox', locale='en-US', extension=None): | |||
| application='firefox', locale='en-US', extension=None, | |||
| username=None, password=None): | |||
There was a problem hiding this comment.
I would use a dict here with authentication as name. It should have username and password as properties.
|
Looks fine otherwise. |
|
Updated |
| CHUNK = 16 * 1024 | ||
| with open(tmp_file, 'wb') as f: | ||
| for chunk in iter(lambda: r.read(CHUNK), ''): | ||
| f.write(chunk) |
There was a problem hiding this comment.
Nice! Have found the same on stackoverflow. Ii just wonder if 16KB is the right amount of data we want to read-in for each iteration or if we should better go with 64KB, which I think is a better size.
There was a problem hiding this comment.
we should better go with 64KB, which I think is a better size
Out of interest, what is this based on?
There was a problem hiding this comment.
Previous experience. But it's a couple of years back so things could have changed. Values too small could cause an overhead while larger values could block the thread. I think we can leave it for now and figure out performance improvements later.
|
So please merge to patch to master. It looks fine! |
|
Raised issue #43 to take care of the default=None items. |
|
Landed in commit 44efef0 |
Fixes issue #36