Join GitHub today
GitHub is home to over 20 million developers working together to host and review code, manage projects, and build software together.
encoding not respected on read_msgpack #10581
Comments
jreback
added Bug Unicode Msgpack
labels
Jul 15, 2015
jreback
added this to the
0.17.0
milestone
Jul 15, 2015
|
On Py2.7, even after making this change, this surprisingly raises a UnicodeDecodeError in msgpack.cpp:
but i'm having trouble stepping through as my PyCharm environment crashes on me when inspecting. |
kawochen
referenced
this issue
Jul 28, 2015
Merged
BUG: GH10581 where read_msgpack does not respect encoding #10686
jreback
closed this
in #10686
Aug 18, 2015
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
ruidc commentedJul 15, 2015
as discussed on https://groups.google.com/forum/#!topic/pydata/ngROaML_hLI
encoding does not seem to be respected on reading a msgpack, below i am expecting to get back what
I put in as utf8
in stepping through, part of the problem seems to be that in the call to unpack on https://github.com/pydata/pandas/blob/master/pandas/io/packers.py#L134 that there is no encoding argument passed and so it defaults to latin1 in https://github.com/pydata/pandas/blob/master/pandas/io/packers.py#L558
changing L134 to :
and passing the encoding like:
makes it work for me, however i don't have en environment set up to submit this as a pull request via GH, and we're still using 0.14.1 due to compatibility issues.