pa-philadelphia María Quiñones-Sánchez page contains unicode chars

[rchekaluk]

Similar to opengovernment/opengovernment-local#9

Error is generated when scraping:

saving Councilmember O'Brien 522f2ce812a96023f0e70e95
saving Councilmember Oh 522f2ce812a96023f0e70e96
saving Councilmember O'Neill 522f2ce812a96023f0e70e97
saving Traceback (most recent call last):
  File "./import.py", line 89, in <module>
    print 'saving', member['Person Name'], council_member_id
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf1' in position 17: ordinal not in range(128)

[walter]

@derekeder can you take a look at this ASAP, please. This is blocking us for a server set up.

[derekeder]

@rchekaluk could you post a full traceback for this error?

[derekeder]

@rchekaluk actually, it looks like an error with the print statement. Will push up a fix shortly.

print str.decode('utf-8')

[derekeder]

fixed with 65c4495

@rchekaluk give it another try

[rchekaluk]

Typo, this fix fails per below. I think you meant "encode" not "decode" (see opengovernment/opengovernment-local#9 (comment)); I experimented with "encode" manually and it resolved the issue.

saving Councilmember O'Brien 5232113d12a96075ffde86c7
saving Councilmember Oh 5232113d12a96075ffde86c8
saving Councilmember O'Neill 5232113d12a96075ffde86c9
saving
Traceback (most recent call last):
  File "/u/apps/legistar-scrape-staging/import.py", line 89, in <module>
    print 'saving', member['Person Name'].decode('utf-8'), council_member_id
  File "/u/apps/virtualenvs/legistar/lib64/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf1' in position 17: ordinal not in range(128)

[derekeder]

@rchekaluk hmm strange. not quite sure why that worked. uploaded a fix with encode 22495fe

[rchekaluk]

Success, thx.