pa-philadelphia María Quiñones-Sánchez page contains unicode chars

[rchekaluk]

Error when Billy scrapes this page:

02:05:18 INFO scrapelib: GET -
http://philadelphiacitycouncil.net/council-members/councilwoman-maria-d-quinones-sanchez-7th-district/councilwoman-maria-d-quinones-sanchez-contact/
Traceback (most recent call last):
File "/u/apps/virtualenvs/billy/src/billy/billy/ext/ansistrm.py", line 56, in emit
stream.write(message)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf1' in position 45:
ordinal not in range(128)
Logged from file init.py, line 177

Here are the obvious non-ascii chars I can see, indeed the first one is Unicode point F1:

• ñ http://www.fileformat.info/info/unicode/char/search.htm?q=%C3%B1&preview=entity
• á http://www.fileformat.info/info/unicode/char/search.htm?q=%C3%A1&preview=entity
• í http://www.fileformat.info/info/unicode/char/search.htm?q=%C3%AD&preview=entity

Not certain, but I'm wondering if Billy might be constrained to the ascii character set:

billy$ find . -type f -print | xargs grep ascii | egrep -v git
./billy/importers/bills.py:            r.encode('ascii', 'replace') for r in remaining]))
./billy/importers/committees.py:            logger.debug("No matches for %s" % member['name'].encode('ascii',
./billy/web/api/emitters.py:                           ensure_ascii=False)
./billy/web/api/emitters.py:        return obj.encode("ascii", "replace")
./billy/scrape/bills.py:        return filename.encode('ascii', 'replace')
./billy/scrape/legislators.py:        return filename.encode('ascii', 'replace')
./billy/scrape/legislators.py:        return filename.encode('ascii', 'replace')
./billy/utils/fulltext.py:        text = text.encode('ascii', 'ignore')
./billy/utils/fulltext.py:        text = text.decode('utf8', 'ignore').encode('ascii', 'ignore')

[rchekaluk]

Fixed in billy opengovernment/billy@8fd1500
Reference http://stackoverflow.com/questions/9942594/unicodeencodeerror-ascii-codec-cant-encode-character-u-xa0-in-position-20/9942885#9942885