-
-
Notifications
You must be signed in to change notification settings - Fork 40
pickle file not readable on Mac #175
Comments
A couple of notes:
|
Hi,
Thanks a lot.. It is a pity that your project is no longer maintained. Sounds like a very sensible one..
I have developed several version of my own pickles and it turned out that my problem was due to incompatible versions of matplotib and had nothing to do with the actual contant of the file..
Best..
Adam
From: Martin Durant <notifications@github.com>
Reply-To: dask/hdfs3 <reply@reply.github.com>
Date: Sunday, July 28, 2019 at 11:41
To: dask/hdfs3 <hdfs3@noreply.github.com>
Cc: Adam Para <para@fnal.gov>, Author <author@noreply.github.com>
Subject: Re: [dask/hdfs3] pickle file not readable on Mac (#175)
A couple of notes:
* this project is no longer actively maintained, so I would not recommend it any more. However if it works for you, and solves your problem, then it may be fine. You might be interested in pyarrow's hdfs, and the independent webhdfs drivers, both of which are available through fsspec<https://urldefense.proofpoint.com/v2/url?u=https-3A__filesystem-2Dspec.readthedocs.io_en_latest_api.html-23built-2Din-2Dimplementations&d=DwMCaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=6SO_4sQi2JZ-pUj2QK9TQg&m=esYdpM1xupLMlVZ3XOpf6pXMvrRb6fZoGsF-qpxnyBI&s=1IrM2pQHFXf3vTgJnRGUGrdQMG3NBPNMjotTu-e5f8Q&e=>
* pickle is very flexible and useful for temporary keeping of python objects. However it is not meant for long-term storage and certainly not between different platforms/installations. To get pickle to work between machines, you need to be very careful about matching versions.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_dask_hdfs3_issues_175-3Femail-5Fsource-3Dnotifications-26email-5Ftoken-3DACCV5O6Y3OOHIIIOG5G4AKTQBXD4TA5CNFSM4IHLH5I2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD27CEOY-23issuecomment-2D515777083&d=DwMCaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=6SO_4sQi2JZ-pUj2QK9TQg&m=esYdpM1xupLMlVZ3XOpf6pXMvrRb6fZoGsF-qpxnyBI&s=A-3noSvUCb9nru71kBiD1gQU1ipsUgLkOh4WEH-l4PI&e=>, or mute the thread<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ACCV5O3QIYAALBL5IZZFK4TQBXD4TANCNFSM4IHLH5IQ&d=DwMCaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=6SO_4sQi2JZ-pUj2QK9TQg&m=esYdpM1xupLMlVZ3XOpf6pXMvrRb6fZoGsF-qpxnyBI&s=cCSxvySOd4oo9AwVnHQS0XejX7jRb43DEU-prGA77rk&e=>.
|
We found that pyarrow's implementation served for most people's needs, and that building the library to correctly identify and use all the possible security modes in HDFS proved too difficult. |
I have a pickle file created ('wb') on the linux machine. It reads back fine using Python 2.7.13 |Anaconda 2.3.0 (64-bit)| (default, Dec 20 2016, 23:09:15)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
I copy (via scp) the pickle file to MacOs (High Sierra). The copy seems to be identical (od -c) to the original. An attempt to read it back with
Python 2.7.16 |Anaconda custom (x86_64)| (default, Mar 14 2019, 16:24:02)
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
produces:
Traceback (most recent call last):
File "read_plot_plots.py", line 29, in
fig_handle = pl.load(open(plt_file,'rb'))
File "/Users/para/anaconda/lib/python2.7/pickle.py", line 1384, in load
return Unpickler(file).load()
File "/Users/para/anaconda/lib/python2.7/pickle.py", line 864, in load
dispatchkey
File "/Users/para/anaconda/lib/python2.7/pickle.py", line 1223, in load_build
setstate(state)
File "/Users/para/anaconda/lib/python2.7/site-packages/matplotlib/transforms.py", line 1693, in setstate
self.input_dims = state['input_dims']
KeyError: u'input_dims'
Any helpful suggestions will be appreciated
Adam Para
The text was updated successfully, but these errors were encountered: