Cannot plot bar graph with dates: "TypeError: ufunc subtract cannot use operands with types dtype('<M8[ns]') and dtype('float64')"

[derekhuizhang]

Bug report

Bug summary

Unable to plot bar graph. Plotting a line graph works fine.

Code for reproduction

from boto3.dynamodb.conditions import Key, Attr
import pandas as pd
import json
import decimal 
import matplotlib.pyplot as plt
import matplotlib.dates as mdate
from datetime import datetime

def timestamp_to_datetime(raw):
    return datetime.fromtimestamp(raw/1000)

df = pd.DataFrame(table.query(KeyConditionExpression=Key('partition_key').eq(name))['Items'])
df['date'] = df['timestamp'].apply(timestamp_to_datetime) 

plt.figure(figsize=(20,10))
plt.bar(df['date'], df['activity'].values)
plt.show();

Actual outcome

TypeError                                 Traceback (most recent call last)
<ipython-input-13-f28db9680a8e> in <module>()
     26 
     27 plt.figure(figsize=(20,10))
---> 28 plt.bar(df['date'], df['activity'].values)

c:\python27\lib\site-packages\matplotlib\pyplot.pyc in bar(*args, **kwargs)
   2773                       mplDeprecation)
   2774     try:
-> 2775         ret = ax.bar(*args, **kwargs)
   2776     finally:
   2777         ax._hold = washold

c:\python27\lib\site-packages\matplotlib\__init__.pyc in inner(ax, *args, **kwargs)
   1865                         "the Matplotlib list!)" % (label_namer, func.__name__),
   1866                         RuntimeWarning, stacklevel=2)
-> 1867             return func(ax, *args, **kwargs)
   1868 
   1869         inner.__doc__ = _add_data_doc(inner.__doc__,

c:\python27\lib\site-packages\matplotlib\axes\_axes.pyc in bar(self, *args, **kwargs)
   2262         if align == 'center':
   2263             if orientation == 'vertical':
-> 2264                 left = x - width / 2
   2265                 bottom = y
   2266             elif orientation == 'horizontal':

TypeError: ufunc subtract cannot use operands with types dtype('<M8[ns]') and dtype('float64')

Expected outcome

Expected: a plot of the (date, activity) in bar graph form. Changing plt.bar to plt.plot, as shown below, works perfectly fine:

from boto3.dynamodb.conditions import Key, Attr
import pandas as pd
import json
import decimal 
import matplotlib.pyplot as plt
import matplotlib.dates as mdate
from datetime import datetime

def timestamp_to_datetime(raw):
    return datetime.fromtimestamp(raw/1000)

df = pd.DataFrame(table.query(KeyConditionExpression=Key('partition_key').eq(name))['Items'])
df['date'] = df['timestamp'].apply(timestamp_to_datetime) 

plt.figure(figsize=(20,10))
**plt.plot(df['date'], df['activity'].values)**
plt.show();

Matplotlib version

• Operating system: Windows 10
• Matplotlib version: 2.2.3
• Matplotlib backend (print(matplotlib.get_backend())): module://ipykernel.pylab.backend_inline
• Python version: 2.7

[jklymak]

Can you make this self contained? Also, I suspect this has been fixed. But see #12862 for potentially similar issue.

[derekhuizhang]

Try this:

import pandas as pd
import json
import decimal 
import matplotlib.pyplot as plt
from matplotlib.backends.backend_pdf import PdfPages
import matplotlib.dates as mdate
from datetime import datetime

import matplotlib

def timestamp_to_datetime(raw):
    return datetime.fromtimestamp(raw/1000)

df = pd.DataFrame([{
    'timestamp': 1500854400000,
    'activity': 5},
{
    'timestamp': 1500940800000,
    'activity': 6
}])
df['date'] = df['timestamp'].apply(timestamp_to_datetime) 

plt.figure(figsize=(20,10))
plt.bar(df['date'], df['activity'])
plt.xlabel('date')
plt.ylabel('activity')
plt.legend()
plt.show();

I don't think it has been fixed. I'm using the latest version (2.2.3) and it's the same issue, but it doesn't work for me.

[jklymak]

I’ll look at it, but 3.0.2 is the latest version.

[jklymak]

The default width for a bar is a float width=0.8, which as pointed out in the error message can't be added to a numpy.datetime64 in a sensible manner. The fix is to specify the width in the appropriate units:

plt.bar(df['date'], df['activity'], width=np.timedelta64(12, 'h'), )

I think there is a question here about whether Matplotlib should do something about this - i.e. if the data on the xaxis is not a float, what width should bar use by default? But then the question is what value do you ascribe to the width?

I think a second question is if matplotlib could supply a better error if the xaxis units are not numbers and width is not specified. My tendency would be to do this rather than try to guess what the user wants.

[timhoffm]

+1 for the better error message. Probably something like

try:
    left = x - width / 2
except TypeError as e:
    raise ValueError('x and width have incompatible types') from e

[jklymak]

Yeah, I looked at doing that quickly. My only hesitation is if that is particularly more helpful than:

c:\python27\lib\site-packages\matplotlib\axes\_axes.pyc in bar(self, *args, **kwargs)
   2262         if align == 'center':
   2263             if orientation == 'vertical':
-> 2264                 left = x - width / 2
   2265                 bottom = y
   2266             elif orientation == 'horizontal':

TypeError: ufunc subtract cannot use operands with types dtype('<M8[ns]') and dtype('float64')

[timhoffm]

Well, maybe if we do

Parameters x and width have incompatible types

That explicitly tells the user to go and change the parameters. Whereas in the original traceback, he first has to realize that x - width is the problem and then has to know that width comes from a parameter (which he might not be aware of if he's using the default).

[jklymak]

@timhoffm Thats a good point, and agree with that as a good reason for a better error message

[derekhuizhang]

import pandas as pd
import json
import decimal 
import matplotlib.pyplot as plt
from matplotlib.backends.backend_pdf import PdfPages
import matplotlib.dates as mdate
from datetime import datetime

import matplotlib

def timestamp_to_datetime(raw):
    return datetime.fromtimestamp(raw/1000)

df = pd.DataFrame([{
    'timestamp': 1500854400000,
    'activity': 5},
{
    'timestamp': 1500940800000,
    'activity': 6
}])
df['date'] = df['timestamp'].apply(timestamp_to_datetime) 

plt.figure(figsize=(20,10))
plt.bar(df['date'], df['activity'], width=np.timedelta64(24, 'h'))
plt.xlabel('date')
plt.ylabel('activity')
plt.legend()
plt.show();

This works. Thanks so much!