Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update scrapyapperyio.py #1

Merged
merged 3 commits into from
May 30, 2017
Merged

Conversation

dragonabyss
Copy link
Contributor

Encode response.body with utf-8, otherwise the json.loads function will fail.

encode response.body with utf-8, otherwise the json.loads function will fail
urlencode function doesn't exist in urllib, but in urlib.parse in python3. Now it works well for both python2 and python3.
@lookfwd
Copy link
Collaborator

lookfwd commented May 26, 2017

Thanks @dragonabyss! Will have a look at it in the next 2 days!

@@ -55,7 +58,7 @@ def process_item(self, item, spider):
def extract_session(response):
if response.status != 200:
raise RuntimeError("Unable to login: %s" % response.body)
self._session = json.loads(response.body)['sessionToken']
self._session = json.loads(str(response.body,encoding='utf-8'))['sessionToken']
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It all looks perfect - I would add a space after the ',' for pep8'ish (although the rest of the file isn't right now)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes,and I have added a space after comma.

add a space after comma
@lookfwd lookfwd merged commit dafda0d into scalingexcellence:master May 30, 2017
@lookfwd
Copy link
Collaborator

lookfwd commented May 30, 2017

Thanks a lot! :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants