DataFrame.plot overrides previous plots if the frame's index is DatetimeIndex

[esvhd]

Environment is Anaconda 2.3, using python 2.7.10. Pandas version 0.16.2.

Here's the code to reproduce this problem in IPython Notebook 3.2, only df['col3'] is plotted.

When sharing an axis the last plot() seems to override whatever was plotted before. Only the last plotted item is shown in figure.

import pandas as pd
import bumpy as np
import matplotlib.pyplot as plt
%matplotlib inline

drange = pd.date_range(start='2015-07-01', end='2015-07-10', freq='D')
df = pd.DataFrame(data=np.random.random_sample((10, 3)), index=drange, columns=['col1', 'col2', 'col3'])
fig, ax = plt.subplots()
plt.hold(True) # when the index is just integer type, this will make it work.
df[['col1', 'col2']].plot(kind='bar', ax=ax, stacked=True)
df[['col3']].plot(ax=ax, color='r', marker='o', ls='-', alpha=.7) # This overrides the previous plot
# if you use this line below instead of the line above, both bars and lines will show up:
#ax.plot(df[['col3']], color='r', marker='o', ls='-', alpha=.7)
ax.grid(True)
ax.legend(loc='best')

[jorisvandenbossche]

If I run your code, the last plot does not override it (the plotted bars are still there if you zoom in to the right place).
But you hit the problem that the barplot implementation in pandas does not 'really' support datetimes (it does plot it, but the actual x-values are not datetimes).

So you end up with two incompatible x-values. The barplot will plot it for x axis values 1, 2, 3, 4, ..., while the datetime plot with df[['col3']].plot() uses the actual datetime values for x axis values (this means either converted to Periods, or using the matplotlib float representation -> but in any case, both mean that the values are way above 1000).

[jorisvandenbossche]

Related issues #1918 and #8803

[esvhd]

Thanks @jorisvandenbossche.

Just to add, my hack to make it work is to use ax.plot() for the second set of lines. See code with comment below, only one line of difference. x-axis shows datetime this time..

import pandas as pd
import bumpy as np
import matplotlib.pyplot as plt
%matplotlib inline

drange = pd.date_range(start='2015-07-01', end='2015-07-10', freq='D')
df = pd.DataFrame(data=np.random.random_sample((10, 3)), index=drange, columns=['col1', 'col2', 'col3'])
fig, ax = plt.subplots()
plt.hold(True) # when the index is just integer type, this will make it work.
df[['col1', 'col2']].plot(kind='bar', ax=ax, stacked=True)
#df[['col3']].plot(ax=ax, color='r', marker='o', ls='-', alpha=.7) # This overrides the previous plot

# if you use this line below instead of the line above, both bars and lines will show up:
ax.plot(df[['col3']], color='r', marker='o', ls='-', alpha=.7)

ax.grid(True)
ax.legend(loc='best')

[dunkinnear]

Thanks @esvhd for the hack, but it didn't work for me, I'm using Jupyter Notebook Version 4.2.0 with Python 2.7.11. I added to the hack to use matplotlib to plot the bar charts and the line, this was the only way I could get it to work.

import pandas as pd
import matplotlib
import matplotlib.pyplot as plt
matplotlib.style.use('ggplot')
%matplotlib inline

df = pd.read_excel('data.xlsx', index_col='Date')
fig, ax = plt.subplots(figsize=(15,10)) 
ax.hold(True)
g = df.groupby(pd.TimeGrouper("M"))
width=10

ax.bar(g.sum().index, g['Money Out'].sum(),width=-width, color='r', label='Money Out')
ax.bar(g.sum().index, g['Money in'].sum(),width=-width,color='g', label='Money In', bottom=g['Money Out'].sum())
ax.plot(df[['Balance']], color='black', lw=1, ls='-',label='Balance')  # Workaround for Pandas issue plotting datetime line on same plot as barchart

ax.legend(loc='best')

[ChristopherShort]

@dunkinnear hack works only if the index is a DatetimeIndex.

Using a PeriodIndex throws the error TypeError: Axis must havefreqset to convert to Periods

But that did give me a trace stack to try and work through tomorrow (something I didn't have in my examples)

I may be back with questions :-)