Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: to_numeric casts floats incorrectly #36502

Closed
1 task
dcsaba89 opened this issue Sep 20, 2020 · 3 comments
Closed
1 task

BUG: to_numeric casts floats incorrectly #36502

dcsaba89 opened this issue Sep 20, 2020 · 3 comments
Labels

Comments

@dcsaba89
Copy link

dcsaba89 commented Sep 20, 2020

  • [ X ] I have checked that this issue has not already been reported.

  • [ X ] I have confirmed this bug exists on the latest version of pandas.

  • (optional) I have confirmed this bug exists on the master branch of pandas.


Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

df = pd.DataFrame({'A': ['0.09', '0.63', '0.121909', '0.117863']})
df['B'] = pd.to_numeric(df['A'])
df['C'] = df['A'].astype(float)

Problem description

When floats as strings are passed to a column of a DataFrame, pd.to_numeric casts some of the given strings incorrectly.

In the above case df['C'] and df['A'] should have exactly the same values, but they differ in some decimals:

df['B'] = (0, 0.09) (1, 0.63) (2, 0.12190899999999999) (3, 0.11786300000000001)
df['C'] = (0, 0.09) (1, 0.63) (2, 0.121909) (3, 0.117863)

df['C'] is correct but df['B'] is incorrect.

The above issue does not occur when the same input is passed as list
w = pd.to_numeric(['0.09', '0.63', '0.121909', '0.117863'])

[0.09 0.63 0.121909 0.117863]

[this should explain why the current behaviour is a problem and why the expected output is a better solution]

Expected Output

Output of pd.show_versions()

INSTALLED VERSIONS

commit : 2a7d332
python : 3.8.5.final.0
python-bits : 32
OS : Windows
OS-release : 10
Version : 10.0.19041
machine : AMD64
processor : Intel64 Family 6 Model 142 Stepping 12, GenuineIntel
byteorder : little
LC_ALL : None
LANG : None
LOCALE : English_United States.1252

pandas : 1.1.2

@dcsaba89 dcsaba89 added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Sep 20, 2020
@asishm
Copy link
Contributor

asishm commented Sep 20, 2020

I believe this is related to #36451

In that case #36464 should fix this

@dsaxton
Copy link
Member

dsaxton commented Sep 20, 2020

Hmm, I think this may even be fixed on master:

[ins] In [1]: import pandas as pd
         ...:
         ...:
         ...: df = pd.DataFrame({'A': ['0.09', '0.63', '0.121909', '0.117863']})
         ...: df['B'] = pd.to_numeric(df['A'])
         ...: df['C'] = df['A'].astype(float)
         ...:
         ...: assert (df['B'] == df['C']).all()
         ...:

[ins] In [2]: pd.__version__
Out[2]: '1.2.0.dev0+453.g15539fa62'

@dsaxton dsaxton removed the Needs Triage Issue that has not been reviewed by a pandas team member label Sep 20, 2020
@dsaxton
Copy link
Member

dsaxton commented Sep 20, 2020

Appears this was fixed / tested for in #36149 so closing as resolved

@dsaxton dsaxton closed this as completed Sep 20, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants