Skip to content
This repository has been archived by the owner on Feb 10, 2021. It is now read-only.

pickle file not readable on Mac #175

Closed
adampara opened this issue Jul 28, 2019 · 3 comments
Closed

pickle file not readable on Mac #175

adampara opened this issue Jul 28, 2019 · 3 comments

Comments

@adampara
Copy link

I have a pickle file created ('wb') on the linux machine. It reads back fine using Python 2.7.13 |Anaconda 2.3.0 (64-bit)| (default, Dec 20 2016, 23:09:15)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2

I copy (via scp) the pickle file to MacOs (High Sierra). The copy seems to be identical (od -c) to the original. An attempt to read it back with
Python 2.7.16 |Anaconda custom (x86_64)| (default, Mar 14 2019, 16:24:02)
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
produces:

Traceback (most recent call last):
File "read_plot_plots.py", line 29, in
fig_handle = pl.load(open(plt_file,'rb'))
File "/Users/para/anaconda/lib/python2.7/pickle.py", line 1384, in load
return Unpickler(file).load()
File "/Users/para/anaconda/lib/python2.7/pickle.py", line 864, in load
dispatchkey
File "/Users/para/anaconda/lib/python2.7/pickle.py", line 1223, in load_build
setstate(state)
File "/Users/para/anaconda/lib/python2.7/site-packages/matplotlib/transforms.py", line 1693, in setstate
self.input_dims = state['input_dims']
KeyError: u'input_dims'

Any helpful suggestions will be appreciated

Adam Para

@martindurant
Copy link
Member

A couple of notes:

  • this project is no longer actively maintained, so I would not recommend it any more. However if it works for you, and solves your problem, then it may be fine. You might be interested in pyarrow's hdfs, and the independent webhdfs drivers, both of which are available through fsspec

  • pickle is very flexible and useful for temporary keeping of python objects. However it is not meant for long-term storage and certainly not between different platforms/installations. To get pickle to work between machines, you need to be very careful about matching versions.

@adampara
Copy link
Author

adampara commented Jul 29, 2019 via email

@martindurant
Copy link
Member

It is a pity that your project is no longer maintained.

We found that pyarrow's implementation served for most people's needs, and that building the library to correctly identify and use all the possible security modes in HDFS proved too difficult.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants