-
-
Notifications
You must be signed in to change notification settings - Fork 270
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Incompatibility between pickled node attributes py2<>py3 #560
Comments
tomkooij
added a commit
to tomkooij/PyTables
that referenced
this issue
Jul 1, 2016
Issue_560.h5 contains a datetime object that was pickled on py2. The pickled string contains non-ascii characters.
tomkooij
added a commit
to tomkooij/PyTables
that referenced
this issue
Jul 1, 2016
issue PyTablesgh-560 is not fixed in python>=3.4
Fixed in #561 |
tomkooij
added a commit
to tomkooij/PyTables
that referenced
this issue
Jul 13, 2016
py2_pickled_dict is a dictionary that contains a numpy array that has illegal (non-ascii) characters when pickled on python 2. It will throw a UnicodeDecodeError when unpickled on python 3, but needs encoding='latin1' instead of encoding='bytes'. To check that the dict is unpickled correctly we check the attribute 's' which is u'just a string'.
tomkooij
added a commit
to tomkooij/PyTables
that referenced
this issue
Jul 13, 2016
py2_pickled_dict is a dictionary that contains a numpy array that has illegal (non-ascii) characters when pickled on python 2. It will throw a UnicodeDecodeError when unpickled on python 3, but needs encoding='latin1' instead of encoding='bytes'. To check that the dict is unpickled correctly we check the attribute 's' which is u'just a string'.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Issue #368 contains a lot of useful background info. Most of the problems reported there were fixed (upstream), but a related issue still remains:
When porting code from py2 to 3 that stores classes in node attributes, I ran into unpickling errors, caused by incompatibility between python 2 and 3 pickles. (Although they are suppossed to be compatible).
Most of these problems were fixed in numpy-1.9 (see also #368) but some classes still fail to be unpickled correctly on python 3. It is easiest demonstrated with
datetime.datetime
objects (see code below).The problem is non-ascii characters in the pickled string. Pickle throws an
UnicodeDecodeError
on these malformed pickled strings in python 3, so it should be possible to fix. I have a fix that I need to test.The text was updated successfully, but these errors were encountered: