ERR: No Error when values argument in pivot_table is not in df.columns #14938

Closed
cdemulde opened this Issue Dec 21, 2016 · 2 comments

Comments

Projects
None yet
3 participants
Contributor

cdemulde commented Dec 21, 2016

Code Sample

import pandas as pd

test = pd.DataFrame({'Date':pd.date_range(pd.datetime.today(), periods=20,freq='M').tolist(),
'occurrenceID':range(0,20)})

test['year'] = test.set_index('eventDate').index.year
test['month'] = test.set_index('eventDate').index.month

heatmap_prep = test.reset_index().pivot_table(index='year',columns='month',values='occurenceID',aggfunc='count')

Problem description

The above code gives no error, while 'occurenceID' is not a dataframe column (as there is a typo, needs to be 'occurrenceID', with two r's). It just produces a dataframe with only an index. Maybe a KeyError should be thrown?

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.5.2.final.0 python-bits: 64 OS: Darwin OS-release: 14.5.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: en_US.UTF-8 LANG: None LOCALE: en_US.UTF-8

pandas: 0.19.1
nose: None
pip: 9.0.1
setuptools: 32.1.0.post20161217
Cython: None
numpy: 1.11.2
scipy: 0.18.1
statsmodels: 0.6.1
xarray: None
IPython: 5.1.0
sphinx: None
patsy: 0.4.1
dateutil: 2.6.0
pytz: 2016.10
blosc: None
bottleneck: None
tables: None
numexpr: None
matplotlib: 1.5.3
openpyxl: None
xlrd: 1.0.0
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.8
boto: None
pandas_datareader: None

Contributor

jreback commented Dec 21, 2016 edited

yep, looks like a bug. These should be validated and a KeyError if not found (with a helpful message, iow which arg its coming from).

jreback added this to the Next Major Release milestone Dec 21, 2016

jreback changed the title from No Error when values argument in pivot_table is not in df.columns to ERR: No Error when values argument in pivot_table is not in df.columns Dec 21, 2016

Contributor

Dr-Irv commented Dec 22, 2016

Looks like the KeyError is generated for when the index and columns arguments have a bad name. So it is only for values that there is an issue.

@Dr-Irv Dr-Irv added a commit to Dr-Irv/pandas that referenced this issue Dec 22, 2016

@Dr-Irv Dr-Irv BUG: Fix #14938 9ba7977

@Dr-Irv Dr-Irv added a commit to Dr-Irv/pandas that referenced this issue Dec 22, 2016

@Dr-Irv Dr-Irv BUG: Fix #14938 c42cd31

Dr-Irv referenced this issue Dec 22, 2016

Merged

ERR: raise on missing values in pd.pivot_table #14965

4 of 4 tasks complete

@jreback jreback modified the milestone: 0.20.0, Next Major Release Dec 23, 2016

@Dr-Irv Dr-Irv added a commit to Dr-Irv/pandas that referenced this issue Dec 23, 2016

@Dr-Irv Dr-Irv ERR: raise on missing values in pd.pivot_table #14938 ff4df5a

jreback closed this in #14965 Dec 23, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment