Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

to_numeric(errors='coerce') broken when using unhashable objects #13324

Closed
RogerThomas opened this issue May 30, 2016 · 1 comment
Closed

to_numeric(errors='coerce') broken when using unhashable objects #13324

RogerThomas opened this issue May 30, 2016 · 1 comment
Labels
Bug Dtype Conversions Unexpected or buggy dtype conversions
Milestone

Comments

@RogerThomas
Copy link
Contributor

RogerThomas commented May 30, 2016

When using pandas.to_numeric with errors='coerce' and non hashable objects in the series/dataframe pandas returns the same series/dataframe as passed in, instead of coercing the non-numeric objects to nans.

python -c "import pandas as pd; s = pd.Series([[13], 1.0, 'apple']); print(pd.to_numeric(s, errors='coerce'))"

The error is due to this line

https://github.com/pydata/pandas/blob/4de83d25d751d8ca102867b2d46a5547c01d7248/pandas/src/inference.pyx#L572

More generally this will affect any call to maybe_convert_numeric that has unhashable objects in the values param.

Suggested fix might be to update this check with something like

if val.__hash__ is not None and val in na_values:
    blah

As there should never be a case where you would expect an unhashable object to be in a set of na_values

output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 2.7.11.final.0
python-bits: 64
OS: Linux
OS-release: 4.4.0-22-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_IE.UTF-8

pandas: 0.18.1
nose: 1.3.7
pip: 8.1.2
setuptools: 19.0
Cython: None
numpy: 1.11.0
scipy: None
statsmodels: None
xarray: None
IPython: None
sphinx: None
patsy: None
dateutil: 2.5.3
pytz: 2016.4
blosc: None
bottleneck: None
tables: None
numexpr: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.8
boto: None
pandas_datareader: None

@jreback
Copy link
Contributor

jreback commented May 30, 2016

yeah, that's prob reasonable.

@jreback jreback added Bug Difficulty Novice Dtype Conversions Unexpected or buggy dtype conversions labels May 30, 2016
@jreback jreback added this to the Next Major Release milestone May 30, 2016
@jreback jreback modified the milestones: 0.18.2, Next Major Release May 31, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Dtype Conversions Unexpected or buggy dtype conversions
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants