test_transformation.py and test_checksum.py fail on Debian on i386 (and others)

[olebole]

Hi,

when compiling for Debian, I have similar problems as the failures shown in #2168 for Fedora.

On x86_64, the tests run fine.
On other platforms, I get:

• coordinates/tests/test_transformations.py:
  • 313: test_composite_static_matrix_transform (i386, powerpc, s390x, powerpc64)
• io/fits/tests/test_checksum.py:
  • 43: TestChecksumFunctions.test_image_create (i386,powerpc, sparc, mipsel, armel, hppa)
  • 56: TestChecksumFunctions.test_nonstandard_checksum (i386, powerpc, sparc, mipsel, armel, hppa)

Since these differ from the failures mentioned in #2168, I create a new issue.
The environment:

platform linux2 -- Python 2.7.6 -- pytest-2.4.0 -- /usr/bin/python2.7

Running tests with Astropy version 0.3.1.
Running tests in astropy /«BUILDDIR»/python-astropy-0.3.1+dfsg/docs.

Platform: Linux-3.2.0-4-amd64-i686-with-debian-jessie-sid

Executable: /usr/bin/python2.7

Full Python Version: 
2.7.6 (default, Feb 26 2014, 06:42:04) 
[GCC 4.8.2]

encodings: sys: ascii, locale: ANSI_X3.4-1968, filesystem: ANSI_X3.4-1968, unicode bits: 20
byteorder: little
float info: dig: 15, mant_dig: 15

Numpy: 1.7.1
Scipy: not available
Matplotlib: 1.3.1
h5py: not available

I used a slightly patched version of astropy-0.3.1. Patches are for python-3.4 and I also still made some failures silent.

[olebole]

Here is the log of test_composite_static_matrix_transform:

    def test_composite_static_matrix_transform():
[...]
        id_mat = np.identity(3)

        id_transform = t.CompositeStaticMatrixTransform(ICRS, ICRS,
                                                        [forwards_45_mat,
                                                        backwards_45_mat])

>       npt.assert_allclose(id_transform.matrix, id_mat)

astropy/coordinates/tests/test_transformations.py:329: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

actual = array([[  1.00000000e+00,   4.26633555e-17,   0.00000000e+00],
       [  4.266....00000000e+00],
       [  0.00000000e+00,   0.00000000e+00,   1.00000000e+00]])
desired = array([[ 1.,  0.,  0.],
       [ 0.,  1.,  0.],
       [ 0.,  0.,  1.]])
rtol = 1e-07, atol = 0, err_msg = '', verbose = True

    def assert_allclose(actual, desired, rtol=1e-7, atol=0,
                        err_msg='', verbose=True):
[...]
        import numpy as np
        def compare(x, y):
            return np.allclose(x, y, rtol=rtol, atol=atol)
        actual, desired = np.asanyarray(actual), np.asanyarray(desired)
        header = 'Not equal to tolerance rtol=%g, atol=%g' % (rtol, atol)
        assert_array_compare(compare, actual, desired, err_msg=str(err_msg),
>                            verbose=verbose, header=header)

/usr/lib/pymodules/python2.7/numpy/testing/utils.py:1179: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

comparison = <function compare at 0x59428a04>
x = array([[  1.00000000e+00,   4.26633555e-17,   0.00000000e+00],
       [  4.266....00000000e+00],
       [  0.00000000e+00,   0.00000000e+00,   1.00000000e+00]])
y = array([[ 1.,  0.,  0.],
       [ 0.,  1.,  0.],
       [ 0.,  0.,  1.]])
err_msg = '', verbose = True
header = 'Not equal to tolerance rtol=1e-07, atol=0'

    def assert_array_compare(comparison, x, y, err_msg='', verbose=True,
                             header=''):
        from numpy.core import array, isnan, isinf, any, all, inf
        x = array(x, copy=False, subok=True)
        y = array(y, copy=False, subok=True)
[...]
        try:
[...]
            if not cond :
>                   raise AssertionError(msg)
E                   AssertionError: 
E                   Not equal to tolerance rtol=1e-07, atol=0
E                   
E                   (mismatch 100.0%)
E                    x: array([[  1.00000000e+00,   4.26633555e-17,   0.00000000e+00],
E                          [  4.26633555e-17,   1.00000000e+00,   0.00000000e+00],
E                          [  0.00000000e+00,   0.00000000e+00,   1.00000000e+00]])
E                    y: array([[ 1.,  0.,  0.],
E                          [ 0.,  1.,  0.],
E                          [ 0.,  0.,  1.]])

/usr/lib/pymodules/python2.7/numpy/testing/utils.py:645: AssertionError

[olebole]

Here is the log of TestChecksumFunctions.test_image_create:

self = <astropy.io.fits.tests.test_checksum.TestChecksumFunctions object at 0x597b398c>

    def test_image_create(self):
        n = np.arange(100)
        hdu = fits.PrimaryHDU(n)
        hdu.writeto(self.temp('tmp.fits'), clobber=True, checksum=True)
        with fits.open(self.temp('tmp.fits'), checksum=True) as hdul:
            assert (hdu.data == hdul[0].data).all()
            assert 'CHECKSUM' in hdul[0].header
            assert 'DATASUM' in hdul[0].header

            if not sys.platform.startswith('win32'):
>               assert hdul[0].header['CHECKSUM'] == 'ZHMkeGKjZGKjbGKj'
E               assert 'ZHLkeHLjZHLjbHLj' == 'ZHMkeGKjZGKjbGKj'
E                 - ZHLkeHLjZHLjbHLj
E                 + ZHMkeGKjZGKjbGKj

astropy/io/fits/tests/test_checksum.py:53: AssertionError

[olebole]

Here is the log of TestChecksumFunctions.test_nonstandard_checksum:

self = <astropy.io.fits.tests.test_checksum.TestChecksumFunctions object at 0x596ea5cc>

    def test_nonstandard_checksum(self):
        hdu = fits.PrimaryHDU(np.arange(10.0 ** 6))
        hdu.writeto(self.temp('tmp.fits'), clobber=True,
                    checksum='nonstandard')
        del hdu
        with fits.open(self.temp('tmp.fits'), checksum='nonstandard') as hdul:
            assert 'CHECKSUM' in hdul[0].header
            assert 'DATASUM' in hdul[0].header

            if not sys.platform.startswith('win32'):
>               assert hdul[0].header['CHECKSUM'] == 'jD4Am942jC48j948'
E               assert 'hA49j948hA48h948' == 'jD4Am942jC48j948'
E                 - hA49j948hA48h948
E                 + jD4Am942jC48j948

astropy/io/fits/tests/test_checksum.py:66: AssertionError

[embray]

Regarding the last two (the TestChecksumFunctions tests, where they read sys.platform.startswith('win32'), really those asserts should be skipped on non-64-bit platforms, where Numpy is using 32-bit by default (leading to different data and thus different checksums). The checksum values being compared against where created from a known-good reference.

[olebole]

@embray I am a bit concerned why the checksums are so similar: 6 resp. 7 characters out of 15 are equal in these cases -- far more than one would expect for different data. So are you sure that this is just due to different data and not some bug in the calculation?

[embray]

@olebole You might have a point there If there was a difference in the data the place to actually look would be the DATASUM keyword and not CHECKSUM. I'll try running these tests again and check whether the DATASUM is actually different too.

I spent quite some time a while back checking the correctness of the checksum calculation, and I'm pretty confident that it is correct. What may have a bug is the checksum encoding algorithm which is a bit strange, and I haven't checked as carefully.

In the meantime it might also be good to rewrite these tests so as to ensure that the data is always the same between platforms. Might be tricky but I think it should be doable.

[sergiopasra]

@olebole If you are seeing in Debian the problems in #2168 it is due to a bug in glibc
https://sourceware.org/bugzilla/show_bug.cgi?id=16724

[embray]

I don't think the problems with checksums are related to #2168, as they seem to be more of an architecture-related issue.

[sergiopasra]

Yes, I was trying to say that if in Debian they are seeing #2168 in addition to the problems in this report, it is due to the glibc bug

[olebole]

Thank you @sergiopasra. I am aware of that. Up to now, the problem didn't show up.

[sergiopasra]

Now I'm seeing this problem in Fedora also when compilling in i686. The following is the build log of an RPM in Fedora 21 i686
http://koji.fedoraproject.org/koji/getfile?taskID=6661230&name=build.log

As you can see the checks fail on TestChecksumFunctions.test_image_create

[sergiopasra]

And the same errors with pyfits 3.2.2

http://koji.fedoraproject.org/koji/getfile?taskID=6678415&name=build.log

[olebole]

These issues are still there in 0.3.2. The coordinates/tests/test_transformations.py failure disappeared on i386, but is still there on powerpc and s390x.

[mdboom]

I'll try to look into this. i386 is an easy platform to investigate because I can use virtualization -- the more obscure platforms that require emulation are much trickier...

[olebole]

I couldn't find the coordinates/tests/test_transformations.py: test_composite_static_matrix_transform in the 0.4rc1 release. Is it gone or moved to another place?

[astrofrog]

@eteq should be able to answer this.

[eteq]

@olebole @astrofrog - That particular test is gone, but the functionality for the static matrix transform is now tested in the test_transform_decos function of coordinates/tests/test_transformations.py .

Note that the implementation has changed quite a bit, so it wouldn't be shocking if this is now gone just by chance.

[olebole]

It actually seems to be gone, I don't get these problems anymore for 0.4rc1. Therefore, I am closing this.

[embray]

Sounds good--thanks @olebole.