-
Notifications
You must be signed in to change notification settings - Fork 25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix broken scrapers-ca scrapers #40
Comments
FYI I am unable to duplicate the "list index out of range" errors on a bunch of these scrapers. ca_on_guelph, ca_on_markham, ca_on_richmond_hill, ca_qc_saint_jerome all complete without errors on my machine. |
ca_qc_mercier page is 403'ing the scraper but loading fine in browser. Maybe some kind of user-agent sniffing going on? Will try to check later. |
same with ca_qc_montreal_est |
Yeah, there are four scrapers that only fail on Heroku, as in the issue description. When you call lxmlize, you can pass a |
I fixed all the Heroku-only failures. They were mostly around the use of things like |
Why is there this difference in interpretation? Is there a difference version of lxml running on heroku? |
The Python package is the same version; maybe the C code is different? I assume only one of the two interpretations is correct, though. |
Indeed. This is very strange. |
The local interpretation is the good one according to http://www.w3.org/TR/1999/REC-xpath-19991116/ |
@matthewleon I added user agent strings for ca_qc_mercier and ca_qc_montreal_est. The scrapers now fail for a different reason ( |
Work through the broken scrapers at http://scrapers.herokuapp.com/
Ignore:
_municipalities
Previous issue content
Reserved for Matthew Leon:
Priority (already in Represent):
The text was updated successfully, but these errors were encountered: