Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pandas._lib.testing.assert_almost_equal seem to not use approximate equality for Series with complex doubles #28235

Closed
oleksandr-pavlyk opened this issue Aug 30, 2019 · 4 comments · Fixed by #36580
Labels
Complex Complex Numbers Testing pandas testing functions or related to the test suite
Milestone

Comments

@oleksandr-pavlyk
Copy link

oleksandr-pavlyk commented Aug 30, 2019

Code Sample, a copy-pastable example if possible

import numpy as np
import pandas as pd
from pandas.util.testing import assert_almost_equal

e1 = np.array([0.5831076580182805, -0.9083518696779991], 'd') 
e2 = np.array([0.5831076580182805, -0.9083518696779992], 'd') 

e1z = np.asanyarray(e1, np.complex128)
e2z = np.asanyarray(e2, np.complex128)

assert_almost_equal(pd.Series(e1), pd.Series(e2)) # gives True as expected

assert_almost_equal(pd.Series(e1z), pd.Series(e2z)) # unexpectedly fails

# both of equivalent tests on NumPy side pass
np.testing.assert_almost_equal(e1, e2)
np.testing.assert_almost_equal(e1z, e2z)

Problem description

When running tests on MacOSX with Pandas 0.25.1, the test pandas.tests.computation.test_eval.TestMathNumExprPandas::test_result_complex128 fails because of this issue.

Expected Output

It is expected that pandas._lib.testing.assert_almost_equal would use np.testing.assert_almost_equal for all np.floating and np.complexfloating dtypes.

Output of pd.show_versions()

In [4]: pd.show_versions()

INSTALLED VERSIONS
------------------
commit           : None
python           : 3.6.9.final.0
python-bits      : 64
OS               : Darwin
OS-release       : 15.4.0
machine          : x86_64
processor        : i386
byteorder        : little
LC_ALL           : en_US.UTF-8
LANG             : en_US.UTF-8
LOCALE           : en_US.UTF-8

pandas           : 0.25.1
numpy            : 1.17.1
pytz             : 2019.1
dateutil         : 2.8.0
pip              : 19.1.1
setuptools       : 41.0.1.post20190716
Cython           : None
pytest           : 3.8.1
hypothesis       : 3.68.0
sphinx           : None
blosc            : None
feather          : None
xlsxwriter       : None
lxml.etree       : None
html5lib         : None
pymysql          : None
psycopg2         : None
jinja2           : None
IPython          : 6.3.1
pandas_datareader: None
bs4              : None
bottleneck       : None
fastparquet      : None
gcsfs            : None
lxml.etree       : None
matplotlib       : 3.1.1
numexpr          : 2.7.0
odfpy            : None
openpyxl         : None
pandas_gbq       : None
pyarrow          : None
pytables         : None
s3fs             : None
scipy            : None
sqlalchemy       : None
tables           : None
xarray           : None
xlrd             : None
xlwt             : None
xlsxwriter       : None
@oleksandr-pavlyk oleksandr-pavlyk changed the title pandas._lib.testing.assert_almost_equal seem to use equal for Series with complex doubles pandas._lib.testing.assert_almost_equal seem to not use approximate equality for Series with complex doubles Aug 30, 2019
@oleksandr-pavlyk
Copy link
Author

ping?

@TomAugspurger
Copy link
Contributor

Sorry for the delay. Looks like a bug (generally, pandas doesn't handle complex values well).

Are you interested in looking into where assert_almost_equal gets this wrong?

@TomAugspurger TomAugspurger added Complex Complex Numbers Testing pandas testing functions or related to the test suite labels Sep 12, 2019
@TomAugspurger TomAugspurger added this to the Contributions Welcome milestone Sep 12, 2019
@oleksandr-pavlyk
Copy link
Author

Yes, I can look into this.

@hwalinga
Copy link
Contributor

Why would pandas role its own (almost) equality checker, when it would just be able to use the one from numpy?

@jreback jreback modified the milestones: Contributions Welcome, 1.2 Sep 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Complex Complex Numbers Testing pandas testing functions or related to the test suite
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants