New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't pickle plots from dataframes with date indexes #18439

Closed
naught101 opened this Issue Nov 23, 2017 · 3 comments

Comments

Projects
None yet
3 participants
@naught101

naught101 commented Nov 23, 2017

Already report this at matplotlib/matplotlib#9828, but it appears that that issue is caused by pandas using a lambda in the format_dateaxis function. I don't know if there is a way around this, but it prevents pandas-produced matplotlib figures from being pickled.

MPL Bug report

Trying to pickle a figure that was produced using pandas.plot(), which includes a datetime index. Not sure if this is a pandas bug, or MPL.

Here's an example:

In [81]: sample
Out[81]: 
time
2002-07-28 08:00:00    0.0
2002-07-28 08:30:00    0.0
2002-07-28 09:00:00    0.0
2002-07-28 09:30:00    0.0
2002-07-28 10:00:00    0.0
2002-07-28 10:30:00    0.0
2002-07-28 11:00:00    0.0
2002-07-28 11:30:00    0.0
2002-07-28 12:00:00    0.0
2002-07-28 12:30:00    0.0
2002-07-28 13:00:00    0.0
2002-07-28 13:30:00    0.0
2002-07-28 14:00:00    0.0
2002-07-28 14:30:00    0.0
2002-07-28 15:00:00    0.0
2002-07-28 15:30:00    0.0
2002-07-28 16:00:00    NaN
2002-07-28 16:30:00    0.0
2002-07-28 17:00:00    0.0
2002-07-28 17:30:00    0.0
Name: Qle_qc, dtype: float64

In [82]: sample.plot()
Out[82]: <matplotlib.axes._subplots.AxesSubplot at 0x7fd5c06a1f60>

In [83]: fig = gcf()

In [84]: pickle.dump(fig, open('test.pkl', 'wb'))
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/data/documents/uni/phd/projects/FluxnetTrafficLights/scripts/analysis/site_predictability.py in <module>()
----> 1 pickle.dump(fig, open('test.pkl', 'wb'))

AttributeError: Can't pickle local object 'format_dateaxis.<locals>.<lambda>'

In [85]: pickle.dump(sample, open('sample.pkl', 'wb'))

I've tried this with a manually constructed DF, and it worked, so I'm not sure why this is happening. I'm including a pickle of the above sample DF for investigation. It is originally a slice from an xarray, from a netcdf dataset.
sample.pkl.gz

Matplotlib version

Using Conda on linux (kubuntu 17.10):

ipython                   6.1.0                    py36_0  
matplotlib                2.1.0            py36hba5de38_0
pandas                    0.20.3                   py36_0  

MPL backend: Qt5Agg
@Licht-T

This comment has been minimized.

Show comment
Hide comment
@Licht-T

Licht-T Nov 25, 2017

Contributor

@naught101 Thanks for reporting this issue.
Your data reproduce the error in my environment, but I cannot create the simple code which reproduce this. Do you have any idea?

Contributor

Licht-T commented Nov 25, 2017

@naught101 Thanks for reporting this issue.
Your data reproduce the error in my environment, but I cannot create the simple code which reproduce this. Do you have any idea?

@Licht-T

This comment has been minimized.

Show comment
Hide comment
@Licht-T

Licht-T Nov 25, 2017

Contributor

Got it. Simple repro. code. The index which length <= 2 does not repro.

import pandas as pd
from matplotlib import pyplot as plt
import numpy
import pickle
%matplotlib inline

idx = pd.DatetimeIndex(['2002-07-28 08:00:00', '2002-07-28 08:30:00', '2002-07-28 09:00:00'])
s = pd.Series([1, 2, 3], index=idx)

s.plot()
fig = plt.gcf()

pickle.dump(fig, open('test.pkl', 'wb'))
Contributor

Licht-T commented Nov 25, 2017

Got it. Simple repro. code. The index which length <= 2 does not repro.

import pandas as pd
from matplotlib import pyplot as plt
import numpy
import pickle
%matplotlib inline

idx = pd.DatetimeIndex(['2002-07-28 08:00:00', '2002-07-28 08:30:00', '2002-07-28 09:00:00'])
s = pd.Series([1, 2, 3], index=idx)

s.plot()
fig = plt.gcf()

pickle.dump(fig, open('test.pkl', 'wb'))
@Licht-T

This comment has been minimized.

Show comment
Hide comment
@Licht-T

Licht-T Nov 25, 2017

Contributor

Seems that this sets the lambda function.

subplot.format_coord = lambda t, y: (

Contributor

Licht-T commented Nov 25, 2017

Seems that this sets the lambda function.

subplot.format_coord = lambda t, y: (

@jreback jreback added this to the 0.21.1 milestone Nov 25, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment