Error "'ascii' codec can't encode character '\xe9' in position 5: ordinal not in range(128)" thrown on file with unicode data

[gvenzl]

When a file includes unicode (or probably non-ascii) data, csv2db throws:

Loading file allCountries.1000.txt.gz
Error while loading file: allCountries.1000.txt.gz
'ascii' codec can't encode character '\xe9' in position 5: ordinal not in range(128)

[cofin]

FYI - I ran into this issue yesterday. In addition to setting the character encoding on the file open, i also had to set the Oracle connection encoding like so:

conn = cx_Oracle.connect(user,
                                     password,
                                     db_name,
                                     encoding="UTF-8", encoding="UTF-8")

[gvenzl]

Thanks @cofin,

Indeed it's an issue with the database connection encoding which defaults to ASCII on Oracle while it defaults to UTF-8 on every other major database (opened an ER for that one to change as well).

[gvenzl]

Fixed.