Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test_psd_csd_equal fails for 12 (but not all) of the test_mlab.spectral_testcases on ppc64 (big-endian) #7798

Closed
AdamWill opened this issue Jan 11, 2017 · 6 comments
Closed

test_psd_csd_equal fails for 12 (but not all) of the test_mlab.spectral_testcases on ppc64 (big-endian) #7798

AdamWill opened this issue Jan 11, 2017 · 6 comments
Labels
status: duplicate

Comments

@AdamWill
Copy link
Contributor

Continuing to work through test failures that happen on Fedora's ppc64 (big-endian), but not on little-endian arches. The last twelve failures are all like this:

======================================================================
FAIL: test_psd_csd_equal (matplotlib.tests.test_mlab.spectral_testcase_Fs4_complex_defaultsided)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILDROOT/python-matplotlib-2.0.0-0.7.rc2.fc26.ppc64/usr/lib64/python2.7/site-packages/matplotlib/tests/test_mlab.py", line 2096, in test_psd_csd_equal
    assert_array_equal(Pxx, Pxy)
  File "/usr/lib64/python2.7/site-packages/numpy/testing/utils.py", line 813, in assert_array_equal
    verbose=verbose, header='Arrays are not equal')
  File "/usr/lib64/python2.7/site-packages/numpy/testing/utils.py", line 739, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not equal

(mismatch 98.4375%)
 x: array([  6.851587e-11,   5.656909e-11,   6.650227e-11,   1.223443e-10,
         1.294211e-10,   4.895139e-11,   4.816325e-11,   2.429269e-10,
         3.889598e-10,   2.063253e-10,   1.297648e-11,   4.368301e-10,...
 y: array([  6.851587e-11 +0.000000e+00j,   5.656909e-11 -7.521779e-30j,
         6.650227e-11 +3.815069e-29j,   1.223443e-10 +8.121089e-30j,
         1.294211e-10 -5.664577e-29j,   4.895139e-11 +1.171260e-28j,...

The twelve classes which fail are:

spectral_testcase_Fs4_complex_defaultsided
spectral_testcase_Fs4_complex_onesided
spectral_testcase_Fs4_complex_twosided
spectral_testcase_Fs4_real_defaultsided
spectral_testcase_Fs4_real_onesided
spectral_testcase_Fs4_real_twosided
spectral_testcase_FsAll_complex_defaultsided
spectral_testcase_FsAll_complex_onesided
spectral_testcase_FsAll_complex_twosided
spectral_testcase_FsAll_real_defaultsided
spectral_testcase_FsAll_real_onesided
spectral_testcase_FsAll_real_twosided

AFAICS, it's all the classes that specify fstims as something other than an empty list in their self.createStim call. All the ones that specify it as an empty list pass.

Obviously for each failure the non-matching arrays are different, but the general failure mode is the same - the values seem to be in some way the same, but the second array has these +foo and -foo bits in it that the first array doesn't. I've no idea what those mean. (To add to all my other admissions, this is the point where I admit I have a history degree and know sod all about math. :>)
@AdamWill
Copy link
Contributor Author

@QuLogic @tacaswell

@AdamWill
Copy link
Contributor Author

this and #7797 are the last errors I have on my list.

@AdamWill
Copy link
Contributor Author

I tweaked test_mlab.py to log array.dtype for both arrays, and they differ. For each failure:

Pxx dtype: float64
Pxy dtype: complex128

Pxx comes from psd(), Pxy comes from csd(). The docstrings for both those functions say the return array should be "real valued", which I have a vague feeling means the type of the Pxy (csd()) array is wrong, it shouldn't be complex?

@QuLogic
Copy link
Member

QuLogic commented Jan 11, 2017

I'm not sure if that's what the doc intended, but the implementation of psd is simply calling csd and returning the real value of its result. These should be equal if the complex part is zero.

These tests are pretty sensitive to machine specs, so I'm not sure if this is really a BE problem considering the complex values shown are really small. I think these tests have failed on 32-bit, Windows, etc...

@AdamWill
Copy link
Contributor Author

OK. So yeah, you're right: the difference is that the complex part seems to always be zero on x86_64, but not on ppc. Here's an example, with the test hacked to print both arrays (so we can compare). On ppc64:

======================================================================
FAIL: test_psd_csd_equal (matplotlib.tests.test_mlab.spectral_testcase_FsAll_real_twosided)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILDROOT/python-matplotlib-2.0.0-0.7.rc2.fc26.ppc64/usr/lib64/python2.7/site-packages/matplotlib/tests/test_mlab.py", line 2101, in test_psd_csd_equal
    assert_array_equal(Pxx, Pxy)
  File "/usr/lib64/python2.7/site-packages/numpy/testing/utils.py", line 813, in assert_array_equal
    verbose=verbose, header='Arrays are not equal')
  File "/usr/lib64/python2.7/site-packages/numpy/testing/utils.py", line 739, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not equal

(mismatch 98.4375%)
 x: array([  1.084763e-07,   5.947154e-08,   3.894422e-09,   5.098036e-08,
         1.243062e-07,   9.022498e-08,   8.124808e-09,   4.643260e-08,
         1.732248e-07,   1.748687e-07,   3.730256e-08,   3.213152e-08,...
 y: array([  1.084763e-07 +0.000000e+00j,   5.947154e-08 -2.846132e-25j,
         3.894422e-09 -3.678538e-27j,   5.098036e-08 -2.565235e-25j,
         1.243062e-07 -7.234946e-26j,   9.022498e-08 -1.238325e-25j,...
-------------------- >> begin captured stdout << ---------------------
Point 1: type is complex128
Point 2: type is complex128
Point 3: type is complex128
Point 4: type is complex128
Point 1: type is complex128
Point 2: type is complex128
Point 3: type is complex128
Point 4: type is complex128
Pxx dtype: float64
Pxy dtype: complex128
[  1.08476336e-07   5.94715433e-08   3.89442221e-09   5.09803565e-08
   1.24306211e-07   9.02249833e-08   8.12480827e-09   4.64325967e-08
   1.73224780e-07   1.74868687e-07   3.73025616e-08   3.21315247e-08
   2.56645591e-07   3.78495718e-07   1.64636913e-07   5.50502817e-09
   3.65709905e-07   8.47209264e-07   6.33377638e-07   3.89956087e-08
   4.59946968e-07   1.96725282e-06   2.35947890e-06   6.69111593e-07
   4.65558016e-07   5.47350031e-06   1.10406251e-05   6.96874074e-06
   2.01432479e-06   1.34293786e-04   3.62404739e-03   7.36941138e-02
   1.65148191e-01   7.42871268e-02   4.76928430e-03   6.81582857e-03
   1.67168449e-02   3.17001258e+00   1.45187131e+01   1.25498987e+01
   1.73778874e+00   7.16797793e-06   9.56318662e-04   3.52196028e-04
   3.76048469e-03   4.86274797e-03   1.60639469e-04   8.36738743e-02
   1.02492512e+00   1.04300542e+01   5.06565652e+02   1.59983761e+03
   9.94083799e+02   8.72208818e+01   6.50190955e-01   1.18657843e-02
   4.62702848e-02   2.35390665e-02   5.00328798e-03   8.33620061e-05
   5.78800491e-04   1.10924699e-03   7.69993665e-04   2.31230749e-04
   7.20369572e-06   2.31230749e-04   7.69993665e-04   1.10924699e-03
   5.78800491e-04   8.33620061e-05   5.00328798e-03   2.35390665e-02
   4.62702848e-02   1.18657843e-02   6.50190955e-01   8.72208818e+01
   9.94083799e+02   1.59983761e+03   5.06565652e+02   1.04300542e+01
   1.02492512e+00   8.36738743e-02   1.60639469e-04   4.86274797e-03
   3.76048469e-03   3.52196028e-04   9.56318662e-04   7.16797793e-06
   1.73778874e+00   1.25498987e+01   1.45187131e+01   3.17001258e+00
   1.67168449e-02   6.81582857e-03   4.76928430e-03   7.42871268e-02
   1.65148191e-01   7.36941138e-02   3.62404739e-03   1.34293786e-04
   2.01432479e-06   6.96874074e-06   1.10406251e-05   5.47350031e-06
   4.65558016e-07   6.69111593e-07   2.35947890e-06   1.96725282e-06
   4.59946968e-07   3.89956087e-08   6.33377638e-07   8.47209264e-07
   3.65709905e-07   5.50502817e-09   1.64636913e-07   3.78495718e-07
   2.56645591e-07   3.21315247e-08   3.73025616e-08   1.74868687e-07
   1.73224780e-07   4.64325967e-08   8.12480827e-09   9.02249833e-08
   1.24306211e-07   5.09803565e-08   3.89442221e-09   5.94715433e-08]
[  1.08476336e-07 +0.00000000e+00j   5.94715433e-08 -2.84613150e-25j
   3.89442221e-09 -3.67853773e-27j   5.09803565e-08 -2.56523489e-25j
   1.24306211e-07 -7.23494618e-26j   9.02249833e-08 -1.23832495e-25j
   8.12480827e-09 -4.41647297e-26j   4.64325967e-08 +1.28640673e-25j
   1.73224780e-07 +7.82839520e-25j   1.74868687e-07 -1.46943579e-24j
   3.73025616e-08 +5.21379537e-26j   3.21315247e-08 -9.98879934e-26j
   2.56645591e-07 -2.51932630e-24j   3.78495718e-07 -2.29858511e-25j
   1.64636913e-07 -5.86317580e-25j   5.50502817e-09 -4.23243447e-26j
   3.65709905e-07 +1.30040232e-24j   8.47209264e-07 -3.26768448e-25j
   6.33377638e-07 +5.45296414e-25j   3.89956087e-08 +4.32223125e-26j
   4.59946968e-07 +3.93179029e-24j   1.96725282e-06 -6.57598782e-24j
   2.35947890e-06 -5.00279701e-24j   6.69111593e-07 +6.31484859e-24j
   4.65558016e-07 -3.47348511e-24j   5.47350031e-06 -4.26818828e-24j
   1.10406251e-05 -1.71439234e-23j   6.96874074e-06 +5.55755871e-23j
   2.01432479e-06 -6.88510379e-24j   1.34293786e-04 -2.38275886e-22j
   3.62404739e-03 +6.38276126e-21j   7.36941138e-02 -8.07214423e-19j
   1.65148191e-01 -1.46089506e-21j   7.42871268e-02 +3.15572022e-19j
   4.76928430e-03 -1.20165215e-20j   6.81582857e-03 +3.89715031e-20j
   1.67168449e-02 +3.18169569e-20j   3.17001258e+00 -4.06157720e-18j
   1.45187131e+01 -5.02233450e-17j   1.25498987e+01 -2.48789047e-17j
   1.73778874e+00 +5.23697878e-18j   7.16797793e-06 -3.32615661e-23j
   9.56318662e-04 +5.36109545e-21j   3.52196028e-04 -1.61538639e-21j
   3.76048469e-03 +1.53275621e-21j   4.86274797e-03 -2.29014420e-20j
   1.60639469e-04 -3.67826079e-23j   8.36738743e-02 -1.42742856e-19j
   1.02492512e+00 +9.31338350e-21j   1.04300542e+01 +1.79285444e-17j
   5.06565652e+02 +1.93454351e-15j   1.59983761e+03 -5.13318286e-15j
   9.94083799e+02 -4.24802118e-15j   8.72208818e+01 -4.05823349e-16j
   6.50190955e-01 -5.28488046e-18j   1.18657843e-02 -4.44280200e-21j
   4.62702848e-02 +5.19197134e-19j   2.35390665e-02 -1.21092953e-20j
   5.00328798e-03 +2.04276173e-20j   8.33620061e-05 +2.08657096e-22j
   5.78800491e-04 +1.30496480e-22j   1.10924699e-03 +6.46221628e-21j
   7.69993665e-04 -3.84005170e-22j   2.31230749e-04 -4.97131460e-22j
   7.20369572e-06 +0.00000000e+00j   2.31230749e-04 -5.40178402e-22j
   7.69993665e-04 +3.76491379e-21j   1.10924699e-03 +1.32380389e-21j
   5.78800491e-04 +1.95827034e-21j   8.33620061e-05 +2.29559642e-22j
   5.00328798e-03 +3.65351125e-20j   2.35390665e-02 -2.86116654e-21j
   4.62702848e-02 +9.48337224e-20j   1.18657843e-02 +7.16644187e-21j
   6.50190955e-01 -3.73518979e-18j   8.72208818e+01 -1.75603816e-16j
   9.94083799e+02 +2.03240210e-15j   1.59983761e+03 -8.68121499e-15j
   5.06565652e+02 +3.47930082e-17j   1.04300542e+01 +3.55393363e-17j
   1.02492512e+00 -7.64520243e-19j   8.36738743e-02 +4.13579098e-19j
   1.60639469e-04 +2.83096779e-23j   4.86274797e-03 +2.21521379e-20j
   3.76048469e-03 -6.47180564e-21j   3.52196028e-04 +2.16729533e-22j
   9.56318662e-04 +4.29533140e-21j   7.16797793e-06 +5.42049093e-24j
   1.73778874e+00 -4.57750364e-18j   1.25498987e+01 -4.63249311e-18j
   1.45187131e+01 -8.12700968e-18j   3.17001258e+00 +4.49780027e-18j
   1.67168449e-02 -3.53493090e-21j   6.81582857e-03 -9.70383835e-21j
   4.76928430e-03 -4.42336402e-21j   7.42871268e-02 +2.61551370e-19j
   1.65148191e-01 +1.46089506e-21j   7.36941138e-02 -3.66057376e-19j
   3.62404739e-03 -1.84588763e-22j   1.34293786e-04 +2.44778130e-22j
   2.01432479e-06 +1.23685108e-24j   6.96874074e-06 +6.04691329e-23j
   1.10406251e-05 -3.53044958e-24j   5.47350031e-06 -2.57330120e-23j
   4.65558016e-07 -2.49977581e-24j   6.69111593e-07 -1.31970547e-24j
   2.35947890e-06 -4.38677588e-24j   1.96725282e-06 -4.66502727e-25j
   4.59946968e-07 -1.59908549e-24j   3.89956087e-08 -1.31521424e-25j
   6.33377638e-07 +1.41405591e-24j   8.47209264e-07 +1.45138841e-25j
   3.65709905e-07 +9.36288967e-25j   5.50502817e-09 -5.76442305e-26j
   1.64636913e-07 -3.39777987e-25j   3.78495718e-07 -1.12236668e-25j
   2.56645591e-07 -1.29552949e-24j   3.21315247e-08 -2.02001357e-26j
   3.73025616e-08 -1.56776094e-25j   1.74868687e-07 +6.68081986e-26j
   1.73224780e-07 +3.87427310e-25j   4.64325967e-08 -1.39654500e-25j
   8.12480827e-09 +3.10884567e-26j   9.02249833e-08 -3.38512457e-26j
   1.24306211e-07 +1.06580790e-25j   5.09803565e-08 +2.30058579e-25j
   3.89442221e-09 -2.29149223e-27j   5.94715433e-08 +9.42794257e-27j]

--------------------- >> end captured stdout << ----------------------

----------------------------------------------------------------------

On x86_64 (with the test sabotaged with an assert True == False at the end, so we get the output):

======================================================================
FAIL: test_psd_csd_equal (matplotlib.tests.test_mlab.spectral_testcase_FsAll_real_twosided)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILDROOT/mpl/usr/lib64/python2.7/site-packages/matplotlib/tests/test_mlab.py", line 2103, in test_psd_csd_equal
    assert True == False
AssertionError: 
-------------------- >> begin captured stdout << ---------------------
Point 1: type is complex128
Point 2: type is complex128
Point 3: type is complex128
Point 4: type is complex128
Point 1: type is complex128
Point 2: type is complex128
Point 3: type is complex128
Point 4: type is complex128
Pxx type: float64
Pxy type: complex128
[  1.08476336e-07   5.94715433e-08   3.89442221e-09   5.09803565e-08
   1.24306211e-07   9.02249833e-08   8.12480827e-09   4.64325967e-08
   1.73224780e-07   1.74868687e-07   3.73025616e-08   3.21315247e-08
   2.56645591e-07   3.78495718e-07   1.64636913e-07   5.50502817e-09
   3.65709905e-07   8.47209264e-07   6.33377638e-07   3.89956087e-08
   4.59946968e-07   1.96725282e-06   2.35947890e-06   6.69111593e-07
   4.65558016e-07   5.47350031e-06   1.10406251e-05   6.96874074e-06
   2.01432479e-06   1.34293786e-04   3.62404739e-03   7.36941138e-02
   1.65148191e-01   7.42871268e-02   4.76928430e-03   6.81582857e-03
   1.67168449e-02   3.17001258e+00   1.45187131e+01   1.25498987e+01
   1.73778874e+00   7.16797793e-06   9.56318662e-04   3.52196028e-04
   3.76048469e-03   4.86274797e-03   1.60639469e-04   8.36738743e-02
   1.02492512e+00   1.04300542e+01   5.06565652e+02   1.59983761e+03
   9.94083799e+02   8.72208818e+01   6.50190955e-01   1.18657843e-02
   4.62702848e-02   2.35390665e-02   5.00328798e-03   8.33620061e-05
   5.78800491e-04   1.10924699e-03   7.69993665e-04   2.31230749e-04
   7.20369572e-06   2.31230749e-04   7.69993665e-04   1.10924699e-03
   5.78800491e-04   8.33620061e-05   5.00328798e-03   2.35390665e-02
   4.62702848e-02   1.18657843e-02   6.50190955e-01   8.72208818e+01
   9.94083799e+02   1.59983761e+03   5.06565652e+02   1.04300542e+01
   1.02492512e+00   8.36738743e-02   1.60639469e-04   4.86274797e-03
   3.76048469e-03   3.52196028e-04   9.56318662e-04   7.16797793e-06
   1.73778874e+00   1.25498987e+01   1.45187131e+01   3.17001258e+00
   1.67168449e-02   6.81582857e-03   4.76928430e-03   7.42871268e-02
   1.65148191e-01   7.36941138e-02   3.62404739e-03   1.34293786e-04
   2.01432479e-06   6.96874074e-06   1.10406251e-05   5.47350031e-06
   4.65558016e-07   6.69111593e-07   2.35947890e-06   1.96725282e-06
   4.59946968e-07   3.89956087e-08   6.33377638e-07   8.47209264e-07
   3.65709905e-07   5.50502817e-09   1.64636913e-07   3.78495718e-07
   2.56645591e-07   3.21315247e-08   3.73025616e-08   1.74868687e-07
   1.73224780e-07   4.64325967e-08   8.12480827e-09   9.02249833e-08
   1.24306211e-07   5.09803565e-08   3.89442221e-09   5.94715433e-08]
[  1.08476336e-07+0.j   5.94715433e-08+0.j   3.89442221e-09+0.j
   5.09803565e-08+0.j   1.24306211e-07+0.j   9.02249833e-08+0.j
   8.12480827e-09+0.j   4.64325967e-08+0.j   1.73224780e-07+0.j
   1.74868687e-07+0.j   3.73025616e-08+0.j   3.21315247e-08+0.j
   2.56645591e-07+0.j   3.78495718e-07+0.j   1.64636913e-07+0.j
   5.50502817e-09+0.j   3.65709905e-07+0.j   8.47209264e-07+0.j
   6.33377638e-07+0.j   3.89956087e-08+0.j   4.59946968e-07+0.j
   1.96725282e-06+0.j   2.35947890e-06+0.j   6.69111593e-07+0.j
   4.65558016e-07+0.j   5.47350031e-06+0.j   1.10406251e-05+0.j
   6.96874074e-06+0.j   2.01432479e-06+0.j   1.34293786e-04+0.j
   3.62404739e-03+0.j   7.36941138e-02+0.j   1.65148191e-01+0.j
   7.42871268e-02+0.j   4.76928430e-03+0.j   6.81582857e-03+0.j
   1.67168449e-02+0.j   3.17001258e+00+0.j   1.45187131e+01+0.j
   1.25498987e+01+0.j   1.73778874e+00+0.j   7.16797793e-06+0.j
   9.56318662e-04+0.j   3.52196028e-04+0.j   3.76048469e-03+0.j
   4.86274797e-03+0.j   1.60639469e-04+0.j   8.36738743e-02+0.j
   1.02492512e+00+0.j   1.04300542e+01+0.j   5.06565652e+02+0.j
   1.59983761e+03+0.j   9.94083799e+02+0.j   8.72208818e+01+0.j
   6.50190955e-01+0.j   1.18657843e-02+0.j   4.62702848e-02+0.j
   2.35390665e-02+0.j   5.00328798e-03+0.j   8.33620061e-05+0.j
   5.78800491e-04+0.j   1.10924699e-03+0.j   7.69993665e-04+0.j
   2.31230749e-04+0.j   7.20369572e-06+0.j   2.31230749e-04+0.j
   7.69993665e-04+0.j   1.10924699e-03+0.j   5.78800491e-04+0.j
   8.33620061e-05+0.j   5.00328798e-03+0.j   2.35390665e-02+0.j
   4.62702848e-02+0.j   1.18657843e-02+0.j   6.50190955e-01+0.j
   8.72208818e+01+0.j   9.94083799e+02+0.j   1.59983761e+03+0.j
   5.06565652e+02+0.j   1.04300542e+01+0.j   1.02492512e+00+0.j
   8.36738743e-02+0.j   1.60639469e-04+0.j   4.86274797e-03+0.j
   3.76048469e-03+0.j   3.52196028e-04+0.j   9.56318662e-04+0.j
   7.16797793e-06+0.j   1.73778874e+00+0.j   1.25498987e+01+0.j
   1.45187131e+01+0.j   3.17001258e+00+0.j   1.67168449e-02+0.j
   6.81582857e-03+0.j   4.76928430e-03+0.j   7.42871268e-02+0.j
   1.65148191e-01+0.j   7.36941138e-02+0.j   3.62404739e-03+0.j
   1.34293786e-04+0.j   2.01432479e-06+0.j   6.96874074e-06+0.j
   1.10406251e-05+0.j   5.47350031e-06+0.j   4.65558016e-07+0.j
   6.69111593e-07+0.j   2.35947890e-06+0.j   1.96725282e-06+0.j
   4.59946968e-07+0.j   3.89956087e-08+0.j   6.33377638e-07+0.j
   8.47209264e-07+0.j   3.65709905e-07+0.j   5.50502817e-09+0.j
   1.64636913e-07+0.j   3.78495718e-07+0.j   2.56645591e-07+0.j
   3.21315247e-08+0.j   3.73025616e-08+0.j   1.74868687e-07+0.j
   1.73224780e-07+0.j   4.64325967e-08+0.j   8.12480827e-09+0.j
   9.02249833e-08+0.j   1.24306211e-07+0.j   5.09803565e-08+0.j
   3.89442221e-09+0.j   5.94715433e-08+0.j]

--------------------- >> end captured stdout << ----------------------

----------------------------------------------------------------------

@AdamWill
Copy link
Contributor Author

I also note that there's a patch in the package which disables exactly these tests on aarch64, and with that knowledge, it turns out this is a dupe of #7158 .

This was referenced Jan 11, 2017
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
status: duplicate
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@QuLogic @AdamWill