New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1.0rc1: test failures in io.fits with Numpy dev #3419

Merged
merged 3 commits into from May 29, 2015

Conversation

Projects
None yet
3 participants
@embray
Member

embray commented May 27, 2015

(Note, this is different from #2960 which is a different failure, and in Python 3)

@embray - with 1.0rc1 and the latest Numpy dev version (as of numpy/numpy@a02f5c8) I am getting the following failures in tests for io.fits:

______________________________________________________________________ TestChecksumFunctions.test_ascii_table_data ______________________________________________________________________

self = <astropy.io.fits.tests.test_checksum.TestChecksumFunctions object at 0x117872890>

    def test_ascii_table_data(self):
        a1 = np.array(['abc', 'def'])
        r1 = np.array([11.0, 12.0])
        c1 = fits.Column(name='abc', format='A3', array=a1)
        # This column used to be E format, but the single-precision float lost
        # too much precision when scaling so it was changed to a D
        c2 = fits.Column(name='def', format='D', array=r1, bscale=2.3,
                         bzero=0.6)
        c3 = fits.Column(name='t1', format='I', array=[91, 92, 93])
        x = fits.ColDefs([c1, c2, c3])
        hdu = fits.TableHDU.from_columns(x)
>       hdu.writeto(self.temp('tmp.fits'), clobber=True, checksum=True)

astropy/io/fits/tests/test_checksum.py:171: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <astropy.io.fits.hdu.table.TableHDU object at 0x1169ce610>, name = '/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/fits-test-Z8k9yJ/tmp.fits', output_verify = 'exception'
clobber = True, checksum = True

    def writeto(self, name, output_verify='exception', clobber=False,
                checksum=False):
        """
            Works similarly to the normal writeto(), but prepends a default
            `PrimaryHDU` are required by extension HDUs (which cannot stand on
            their own).
            """

        from .hdulist import HDUList
        from .image import PrimaryHDU

        hdulist = HDUList([PrimaryHDU(), self])
        hdulist.writeto(name, output_verify, clobber=clobber,
>                       checksum=checksum)

astropy/io/fits/hdu/base.py:1712: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = [<astropy.io.fits.hdu.image.PrimaryHDU object at 0x1169cef10>, <astropy.io.fits.hdu.table.TableHDU object at 0x1169ce610>]
fileobj = <astropy.io.fits.file._File <open file '/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/fits-test-Z8k9yJ/tmp.fits', mode 'wb' at 0x1169c4810>>, output_verify = 'exception'
clobber = True, checksum = True

    def writeto(self, fileobj, output_verify='exception', clobber=False,
                checksum=False):
        """
            Write the `HDUList` to a new file.

            Parameters
            ----------
            fileobj : file path, file object or file-like object
                File to write to.  If a file object, must be opened in a
                writeable mode.

            output_verify : str
                Output verification option.  Must be one of ``"fix"``,
                ``"silentfix"``, ``"ignore"``, ``"warn"``, or
                ``"exception"``.  May also be any combination of ``"fix"`` or
                ``"silentfix"`` with ``"+ignore"``, ``+warn``, or ``+exception"
                (e.g. ``"fix+warn"``).  See :ref:`verify` for more info.

            clobber : bool
                When `True`, overwrite the output file if exists.

            checksum : bool
                When `True` adds both ``DATASUM`` and ``CHECKSUM`` cards
                to the headers of all HDU's written to the file.
            """

        if (len(self) == 0):
            warnings.warn("There is nothing to write.", AstropyUserWarning)
            return

        self.verify(option=output_verify)

        # make sure the EXTEND keyword is there if there is extension
        self.update_extend()

        # make note of whether the input file object is already open, in which
        # case we should not close it after writing (that should be the job
        # of the caller)
        closed = isinstance(fileobj, string_types) or fileobj_closed(fileobj)

        # writeto is only for writing a new file from scratch, so the most
        # sensible mode to require is 'ostream'.  This can accept an open
        # file object that's open to write only, or in append/update modes
        # but only if the file doesn't exist.
        fileobj = _File(fileobj, mode='ostream', clobber=clobber)
        hdulist = self.fromfile(fileobj)

        for hdu in self:
>           hdu._prewriteto(checksum=checksum)

astropy/io/fits/hdu/hdulist.py:680: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <astropy.io.fits.hdu.table.TableHDU object at 0x1169ce610>, checksum = True, inplace = False

    def _prewriteto(self, checksum=False, inplace=False):
        if self._has_data:
            self.data._scale_back(
>               update_heap_pointers=not self._manages_own_heap)

astropy/io/fits/hdu/table.py:446: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = rec.array([('abc', 11.0, 91), ('def', 12.0, 92), ('   ', 0.0, 93)], 
          dtype=[('abc', '|S3'), ('def', '|S25'), ('t1', '|S10')]), update_heap_pointers = True

    def _scale_back(self, update_heap_pointers=True):
        """
            Update the parent array, using the (latest) scaled array.

            If ``update_heap_pointers`` is `False`, this will leave all the heap
            pointers in P/Q columns as they are verbatim--it only makes sense to do
            this if there is already data on the heap and it can be guaranteed that
            that data has not been modified, and there is not new data to add to
            the heap.  Currently this is only used as an optimization for
            CompImageHDU that does its own handling of the heap.
            """

        # Running total for the new heap size
        heapsize = 0

        for indx in range(len(self.dtype.names)):
            recformat = self._coldefs._recformats[indx]
            field = super(FITS_rec, self).field(indx)

            # add the location offset of the heap area for each
            # variable length column
            if isinstance(recformat, _FormatP):
                # Irritatingly, this can return a different dtype than just
                # doing np.dtype(recformat.dtype); but this returns the results
                # that we want.  For example if recformat.dtype is 'a' we want
                # an array of characters.
                dtype = np.array([], dtype=recformat.dtype).dtype

                if update_heap_pointers and self._convert[indx] is not None:
                    # The VLA has potentially been updated, so we need to
                    # update the array descriptors
                    field[:] = 0  # reset
                    npts = [len(arr) for arr in self._convert[indx]]

                    field[:len(npts), 0] = npts
                    field[1:, 1] = (np.add.accumulate(field[:-1, 0]) *
                                    dtype.itemsize)
                    field[:, 1][:] += heapsize

                heapsize += field[:, 0].sum() * dtype.itemsize
                # Even if this VLA has not been read or updated, we need to
                # include the size of its constituent arrays in the heap size
                # total

            if self._convert[indx] is None:
                continue

            if isinstance(recformat, _FormatX):
                _wrapx(self._convert[indx], field, recformat.repeat)
                continue

            _str, _bool, _number, _scale, _zero, bscale, bzero, _ = \
                self._get_scale_factors(indx)

            # conversion for both ASCII and binary tables
            if _number or _str:
                column = self._coldefs[indx]
                if _number and (_scale or _zero) and column._physical_values:
                    dummy = self._convert[indx].copy()
                    if _zero:
                        dummy -= bzero
                    if _scale:
                        dummy /= bscale
                    # This will set the raw values in the recarray back to
                    # their non-physical storage values, so the column should
                    # be mark is not scaled
                    column._physical_values = False
                elif _str:
                    dummy = self._convert[indx]
                elif isinstance(self._coldefs, _AsciiColDefs):
                    dummy = self._convert[indx]
                else:
                    continue

                # ASCII table, convert numbers to strings
                if isinstance(self._coldefs, _AsciiColDefs):
                    starts = self._coldefs.starts[:]
                    spans = self._coldefs.spans
                    format = self._coldefs.formats[indx].strip()

                    # The the index of the "end" column of the record, beyond
                    # which we can't write
                    end = super(FITS_rec, self).field(-1).itemsize
                    starts.append(end + starts[-1])

                    if indx > 0:
                        lead = (starts[indx] - starts[indx - 1] -
                                spans[indx - 1])
                    else:
                        lead = 0

                    if lead < 0:
                        warnings.warn(
                            'Column %r starting point overlaps the '
                            'previous column.' % (indx + 1))

                    trail = starts[indx + 1] - starts[indx] - spans[indx]

                    if trail < 0:
                        warnings.warn(
                            'Column %r ending point overlaps the next '
                            'column.' % (indx + 1))

                    # TODO: It would be nice if these string column formatting
                    # details were left to a specialized class, as is the case
                    # with FormatX and FormatP
                    if 'A' in format:
                        _pc = '%-'
                    else:
                        _pc = '%'

                    fmt = ''.join([_pc, format[1:], ASCII2STR[format[0]],
                                   (' ' * trail)])

                    # not using numarray.strings's num2char because the
                    # result is not allowed to expand (as C/Python does).
                    for jdx in xrange(len(dummy)):
                        x = fmt % dummy[jdx]
                        if len(x) > starts[indx + 1] - starts[indx]:
                            raise ValueError(
                                "Value %r does not fit into the output's "
                                "itemsize of %s." % (x, spans[indx]))
                        else:
                            field[jdx] = x
                    # Replace exponent separator in floating point numbers
                    if 'D' in format:
>                       field.replace(encode_ascii('E'), encode_ascii('D'))
E                       AttributeError: 'numpy.ndarray' object has no attribute 'replace'

astropy/io/fits/fitsrec.py:1041: AttributeError
________________________________________________________________________ TestSingleTable.test_read_from_fileobj _________________________________________________________________________

self = <astropy.io.fits.tests.test_connect.TestSingleTable object at 0x116ecd0d0>
tmpdir = local('/private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/pytest-17/test_read_from_fileobj0')

    def test_read_from_fileobj(self, tmpdir):
        filename = str(tmpdir.join('test_read_from_fileobj.fits'))
        hdu = BinTableHDU(self.data)
        hdu.writeto(filename)
        with open(filename, 'rb') as f:
            t = Table.read(f)
>       assert equal_data(t, self.data)
E       assert equal_data(<Table masked=False length=4>\n   a        b             c         \n int64   st...  4.5\n       3        c                6.7\n       4        d                8.9, array([(1, u'a', 2.3), (2, u'b', 4.5), (3, u'c', 6.7), (4, u'd', 8.9)], \n      dtype=[('a', '<i8'), ('b', '<U1'), ('c', '<f8')]))
E        +  where array([(1, u'a', 2.3), (2, u'b', 4.5), (3, u'c', 6.7), (4, u'd', 8.9)], \n      dtype=[('a', '<i8'), ('b', '<U1'), ('c', '<f8')]) = <astropy.io.fits.tests.test_connect.TestSingleTable object at 0x116ecd0d0>.data

astropy/io/fits/tests/test_connect.py:130: AssertionError
_______________________________________________________________________________ TestMultipleHDU.test_read _______________________________________________________________________________

self = <astropy.io.fits.tests.test_connect.TestMultipleHDU object at 0x1178919d0>, tmpdir = local('/private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/pytest-17/test_read0')

    def test_read(self, tmpdir):
        filename = str(tmpdir.join('test_read.fits'))
        self.hdus.writeto(filename)
        with catch_warnings() as l:
            t = Table.read(filename)
        assert len(l) == 1
        assert str(l[0].message).startswith(
            'hdu= was not specified but multiple tables are present, reading in first available table (hdu=1)')
>       assert equal_data(t, self.data1)
E       assert equal_data(<Table masked=False length=4>\n   a        b             c         \n int64   st...  4.5\n       3        c                6.7\n       4        d                8.9, array([(1, u'a', 2.3), (2, u'b', 4.5), (3, u'c', 6.7), (4, u'd', 8.9)], \n      dtype=[('a', '<i8'), ('b', '<U1'), ('c', '<f8')]))
E        +  where array([(1, u'a', 2.3), (2, u'b', 4.5), (3, u'c', 6.7), (4, u'd', 8.9)], \n      dtype=[('a', '<i8'), ('b', '<U1'), ('c', '<f8')]) = <astropy.io.fits.tests.test_connect.TestMultipleHDU object at 0x1178919d0>.data1

astropy/io/fits/tests/test_connect.py:168: AssertionError
________________________________________________________________________ TestMultipleHDU.test_read_with_hdu_1[1] ________________________________________________________________________

self = <astropy.io.fits.tests.test_connect.TestMultipleHDU object at 0x1162e6410>
tmpdir = local('/private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/pytest-17/test_read_with_hdu_1_1_0'), hdu = 1

    @pytest.mark.parametrize('hdu', [1, 'first'])
    def test_read_with_hdu_1(self, tmpdir, hdu):
        filename = str(tmpdir.join('test_read_with_hdu_1.fits'))
        self.hdus.writeto(filename)
        with catch_warnings() as l:
            t = Table.read(filename, hdu=hdu)
        assert len(l) == 0
>       assert equal_data(t, self.data1)
E       assert equal_data(<Table masked=False length=4>\n   a        b             c         \n int64   st...  4.5\n       3        c                6.7\n       4        d                8.9, array([(1, u'a', 2.3), (2, u'b', 4.5), (3, u'c', 6.7), (4, u'd', 8.9)], \n      dtype=[('a', '<i8'), ('b', '<U1'), ('c', '<f8')]))
E        +  where array([(1, u'a', 2.3), (2, u'b', 4.5), (3, u'c', 6.7), (4, u'd', 8.9)], \n      dtype=[('a', '<i8'), ('b', '<U1'), ('c', '<f8')]) = <astropy.io.fits.tests.test_connect.TestMultipleHDU object at 0x1162e6410>.data1

astropy/io/fits/tests/test_connect.py:184: AssertionError
______________________________________________________________________ TestMultipleHDU.test_read_with_hdu_1[first] ______________________________________________________________________

self = <astropy.io.fits.tests.test_connect.TestMultipleHDU object at 0x116f3d390>
tmpdir = local('/private/var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/pytest-17/test_read_with_hdu_1_first_0'), hdu = 'first'

    @pytest.mark.parametrize('hdu', [1, 'first'])
    def test_read_with_hdu_1(self, tmpdir, hdu):
        filename = str(tmpdir.join('test_read_with_hdu_1.fits'))
        self.hdus.writeto(filename)
        with catch_warnings() as l:
            t = Table.read(filename, hdu=hdu)
        assert len(l) == 0
>       assert equal_data(t, self.data1)
E       assert equal_data(<Table masked=False length=4>\n   a        b             c         \n int64   st...  4.5\n       3        c                6.7\n       4        d                8.9, array([(1, u'a', 2.3), (2, u'b', 4.5), (3, u'c', 6.7), (4, u'd', 8.9)], \n      dtype=[('a', '<i8'), ('b', '<U1'), ('c', '<f8')]))
E        +  where array([(1, u'a', 2.3), (2, u'b', 4.5), (3, u'c', 6.7), (4, u'd', 8.9)], \n      dtype=[('a', '<i8'), ('b', '<U1'), ('c', '<f8')]) = <astropy.io.fits.tests.test_connect.TestMultipleHDU object at 0x116f3d390>.data1

astropy/io/fits/tests/test_connect.py:184: AssertionError
__________________________________________________________________________ TestTableFunctions.test_ascii_table __________________________________________________________________________

self = <astropy.io.fits.tests.test_table.TestTableFunctions object at 0x116a2b250>

    def test_ascii_table(self):
        # ASCII table
        a = fits.open(self.data('ascii.fits'))
        ra1 = np.rec.array([
            (10.123000144958496, 37),
            (5.1999998092651367, 23),
            (15.609999656677246, 17),
            (0.0, 0),
            (345.0, 345)], names='c1, c2')
        assert comparerecords(a[1].data, ra1)

        # Test slicing
>       a2 = a[1].data[2:][2:]

astropy/io/fits/tests/test_table.py:254: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = array([(10.123, 37), (5.1999998, 23), (15.61, 17), (0.0, 0), (345.0, 345)], 
 ... 'formats':['S10','S5'], 'offsets':[0,11], 'itemsize':16}).view(numpy.recarray), start = 2
end = 9223372036854775807

    def __getslice__(self, start, end):
>       return self[slice(start, end)]

astropy/io/fits/fitsrec.py:553: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = array([(10.123, 37), (5.1999998, 23), (15.61, 17), (0.0, 0), (345.0, 345)], 
 ... 'formats':['S10','S5'], 'offsets':[0,11], 'itemsize':16}).view(numpy.recarray)
key = slice(2, 9223372036854775807, None)

    def __getitem__(self, key):
        if isinstance(key, string_types):
            return self.field(key)
        elif isinstance(key, (slice, np.ndarray, tuple, list)):
            # Have to view as a recarray then back as a FITS_rec, otherwise the
            # circular reference fix/hack in FITS_rec.field() won't preserve
            # the slice
            subtype = type(self)
>           out = self.view(np.recarray).__getitem__(key).view(subtype)

astropy/io/fits/fitsrec.py:490: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = array([ ('.15610E+02', <read-write buffer ptr 0x10ae9a6aa, size 1 at 0x1171ce6...], 
      dtype=[('a', 'S10'), ('f1', 'V1'), ('b', 'S5')]).view(numpy.recarray)
dtype = <class 'astropy.io.fits.fitsrec.FITS_rec'>, type = None

    def view(self, dtype=None, type=None):
        if dtype is None:
            return ndarray.view(self, type)
        elif type is None:
            try:
                if issubclass(dtype, ndarray):
>                   return ndarray.view(self, dtype)

/Users/tom/Library/Python/2.7/lib/python/site-packages/numpy/core/records.py:534: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <[AttributeError("'NoneType' object has no attribute 'names'") raised in repr()] SafeRepr object at 0x1171d27e8>
obj = array([ ('.15610E+02', <read-write buffer ptr 0x10ae9a6aa, size 1 at 0x1171ce7...], 
      dtype=[('a', 'S10'), ('f1', 'V1'), ('b', 'S5')]).view(numpy.recarray)

    def __array_finalize__(self, obj):
        if obj is None:
            return

        if isinstance(obj, FITS_rec):
            self._convert = obj._convert
            self._heapoffset = obj._heapoffset
            self._heapsize = obj._heapsize
            self._coldefs = obj._coldefs
            self._nfields = obj._nfields
            self._gap = obj._gap
            self._uint = obj._uint
            self.formats = obj.formats
        else:
            # This will allow regular ndarrays with fields, rather than
            # just other FITS_rec objects
            self._nfields = len(obj.dtype.names)
            self._convert = [None] * len(obj.dtype.names)

            self._heapoffset = getattr(obj, '_heapoffset', 0)
            self._heapsize = getattr(obj, '_heapsize', 0)

            self._coldefs = None
            self._gap = getattr(obj, '_gap', 0)
            self._uint = getattr(obj, '_uint', False)

            # Bypass setattr-based assignment to fields; see #86
            self.formats = None

            attrs = ['_convert', '_coldefs', '_gap']
            for attr in attrs:
                if hasattr(obj, attr):
                    value = getattr(obj, attr, None)
                    if value is None:
                        warnings.warn('Setting attribute %s as None' % attr, AstropyUserWarning)
                    setattr(self, attr, value)

            if self._coldefs is None:
>               self._coldefs = ColDefs(self)

astropy/io/fits/fitsrec.py:273: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = ColDefs(
    name = 'a'; format = '10A'
), input = <[AttributeError("'NoneType' object has no attribute 'names'") raised in repr()] SafeRepr object at 0x1171d2b48>, tbtype = None
ascii = False

    def __init__(self, input, tbtype=None, ascii=False):
        """
            Parameters
            ----------

            input : sequence of `Column`, `ColDefs`, other
                An existing table HDU, an existing `ColDefs`, or any multi-field
                Numpy array or `numpy.recarray`.

            **(Deprecated)** tbtype : str, optional
                which table HDU, ``"BinTableHDU"`` (default) or
                ``"TableHDU"`` (text table).
                Now ColDefs for a normal (binary) table by default, but converted
                automatically to ASCII table ColDefs in the appropriate contexts
                (namely, when creating an ASCII table).

            ascii : bool
            """

        from .hdu.table import _TableBaseHDU
        from .fitsrec import FITS_rec

        if isinstance(input, ColDefs):
            self._init_from_coldefs(input)
        elif (isinstance(input, FITS_rec) and hasattr(input, '_coldefs') and
                input._coldefs):
            # If given a FITS_rec object we can directly copy its columns, but
            # only if its columns have already been defined, otherwise this
            # will loop back in on itself and blow up
            self._init_from_coldefs(input._coldefs)
        elif isinstance(input, np.ndarray) and input.dtype.fields is not None:
            # Construct columns from the fields of a record array
>           self._init_from_array(input)

astropy/io/fits/column.py:950: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = ColDefs(
    name = 'a'; format = '10A'
), array = <[AttributeError("'NoneType' object has no attribute 'names'") raised in repr()] SafeRepr object at 0x116bda098>

    def _init_from_array(self, array):
        self.columns = []
        for idx in range(len(array.dtype)):
            cname = array.dtype.names[idx]
            ftype = array.dtype.fields[cname][0]
>           format = self._col_format_cls.from_recformat(ftype)

astropy/io/fits/column.py:981: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

cls = <class 'astropy.io.fits.column._ColumnFormat'>, recformat = dtype('V1')

    @classmethod
    def from_recformat(cls, recformat):
        """Creates a column format from a Numpy record dtype format."""

>       return cls(_convert_format(recformat, reverse=True))

astropy/io/fits/column.py:195: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

format = dtype('V1'), reverse = True

    def _convert_format(format, reverse=False):
        """
        Convert FITS format spec to record format spec.  Do the opposite if
        reverse=True.
        """

        if reverse:
>           return _convert_record2fits(format)

astropy/io/fits/column.py:1931: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

format = dtype('V1')

    def _convert_record2fits(format):
        """
        Convert record format spec to FITS format spec.
        """

        recformat, kind, dtype = _dtype_to_recformat(format)
        shape = dtype.shape
        option = str(dtype.base.itemsize)

        ndims = len(shape)
        repeat = 1
        if ndims > 0:
            nel = np.array(shape, dtype='i8').prod()
            if nel > 1:
                repeat = nel

        if kind == 'a':
            # This is a kludge that will place string arrays into a
            # single field, so at least we won't lose data.  Need to
            # use a TDIM keyword to fix this, declaring as (slength,
            # dim1, dim2, ...)  as mwrfits does

            ntot = int(repeat) * int(option)

            output_format = str(ntot) + 'A'
        elif recformat in NUMPY2FITS:  # record format
            if repeat != 1:
                repeat = str(repeat)
            else:
                repeat = ''
            output_format = repeat + NUMPY2FITS[recformat]
        else:
>           raise ValueError('Illegal format %s.' % format)
E           ValueError: Illegal format |V1.

astropy/io/fits/column.py:1895: ValueError
____________________________________________________________________ TestTableFunctions.test_new_table_from_recarray ____________________________________________________________________

self = <astropy.io.fits.tests.test_table.TestTableFunctions object at 0x11603aa10>

    def test_new_table_from_recarray(self):
        bright = np.rec.array([(1, 'Serius', -1.45, 'A1V'),
                               (2, 'Canopys', -0.73, 'F0Ib'),
                               (3, 'Rigil Kent', -0.1, 'G2V')],
                              formats='int16,a20,float32,a10',
                              names='order,name,mag,Sp')
        hdu = fits.TableHDU.from_columns(bright, nrows=2)

        # Verify that all ndarray objects within the HDU reference the
        # same ndarray.
        assert (id(hdu.data._coldefs.columns[0].array) ==
                id(hdu.data._coldefs._arrays[0]))
        assert (id(hdu.data._coldefs.columns[0].array) ==
                id(hdu.columns.columns[0].array))
        assert (id(hdu.data._coldefs.columns[0].array) ==
                id(hdu.columns._arrays[0]))

        # Ensure I can change the value of one data element and it effects
        # all of the others.
        hdu.data[0][0] = 213

        assert hdu.data[0][0] == 213
        assert hdu.data._coldefs._arrays[0][0] == 213
        assert hdu.data._coldefs.columns[0].array[0] == 213
        assert hdu.columns._arrays[0][0] == 213
        assert hdu.columns.columns[0].array[0] == 213

        hdu.data._coldefs._arrays[0][0] = 100

        assert hdu.data[0][0] == 100
        assert hdu.data._coldefs._arrays[0][0] == 100
        assert hdu.data._coldefs.columns[0].array[0] == 100
        assert hdu.columns._arrays[0][0] == 100
        assert hdu.columns.columns[0].array[0] == 100

        hdu.data._coldefs.columns[0].array[0] = 500
        assert hdu.data[0][0] == 500
        assert hdu.data._coldefs._arrays[0][0] == 500
        assert hdu.data._coldefs.columns[0].array[0] == 500
        assert hdu.columns._arrays[0][0] == 500
        assert hdu.columns.columns[0].array[0] == 500

        hdu.columns._arrays[0][0] = 600
        assert hdu.data[0][0] == 600
        assert hdu.data._coldefs._arrays[0][0] == 600
        assert hdu.data._coldefs.columns[0].array[0] == 600
        assert hdu.columns._arrays[0][0] == 600
        assert hdu.columns.columns[0].array[0] == 600

        hdu.columns.columns[0].array[0] = 800
        assert hdu.data[0][0] == 800
        assert hdu.data._coldefs._arrays[0][0] == 800
        assert hdu.data._coldefs.columns[0].array[0] == 800
        assert hdu.columns._arrays[0][0] == 800
        assert hdu.columns.columns[0].array[0] == 800

        assert (hdu.data.field(0) ==
                np.array([800, 2], dtype=np.int16)).all()
        assert hdu.data[0][1] == 'Serius'
        assert hdu.data[1][1] == 'Canopys'
        assert (hdu.data.field(2) ==
                np.array([-1.45, -0.73], dtype=np.float32)).all()
        assert hdu.data[0][3] == 'A1V'
        assert hdu.data[1][3] == 'F0Ib'

        with ignore_warnings():
            hdu.writeto(self.temp('toto.fits'), clobber=True)

        with fits.open(self.temp('toto.fits')) as hdul:
            assert (hdul[1].data.field(0) ==
                    np.array([800, 2], dtype=np.int16)).all()
>           assert hdul[1].data[0][1] == 'Serius'
E           assert 'Serius              ' == 'Serius'
E             - Serius              
E             + Serius

astropy/io/fits/tests/test_table.py:428: AssertionError
__________________________________________________________________________ TestTableFunctions.test_new_fitsrec __________________________________________________________________________

self = <astropy.io.fits.tests.test_table.TestTableFunctions object at 0x115cfe290>

    def test_new_fitsrec(self):
        """
            Tests creating a new FITS_rec object from a multi-field ndarray.
            """

        h = fits.open(self.data('tb.fits'))
        data = h[1].data
        new_data = np.array([(3, 'qwe', 4.5, False)], dtype=data.dtype)
        appended = np.append(data, new_data).view(fits.FITS_rec)
>       assert repr(appended).startswith('FITS_rec(')
E       assert <built-in method startswith of str object at 0x1162dd2b8>('FITS_rec(')
E        +  where <built-in method startswith of str object at 0x1162dd2b8> = "array([(1, 'abc', 1.1, False), (2, 'xy ', 2.0999999, True),\n (3, 'qwe', 4.5, False)], \n      dtype=[('c1', '>i4'), ('c2', 'S3'), ('c3', '>f4'), ('c4', 'i1')]).view(numpy.recarray)".startswith
E        +    where "array([(1, 'abc', 1.1, False), (2, 'xy ', 2.0999999, True),\n (3, 'qwe', 4.5, False)], \n      dtype=[('c1', '>i4'), ('c2', 'S3'), ('c3', '>f4'), ('c4', 'i1')]).view(numpy.recarray)" = repr(array([(1, 'abc', 1.1, False), (2, 'xy ', 2.0999999, True),\n (3, 'qwe', 4.5, F...('c1', '>i4'), ('c2', 'S3'), ('c3', '>f4'), ('c4', 'i1')]).view(numpy.recarray))

astropy/io/fits/tests/test_table.py:456: AssertionError
_____________________________________________________________________ TestTableFunctions.test_string_column_padding _____________________________________________________________________

self = <astropy.io.fits.tests.test_table.TestTableFunctions object at 0x1169c1810>

    def test_string_column_padding(self):
        a = ['img1', 'img2', 'img3a', 'p']
        s = 'img1\x00\x00\x00\x00\x00\x00' \
            'img2\x00\x00\x00\x00\x00\x00' \
            'img3a\x00\x00\x00\x00\x00' \
            'p\x00\x00\x00\x00\x00\x00\x00\x00\x00'

        acol = fits.Column(name='MEMNAME', format='A10',
                           array=chararray.array(a))
        ahdu = fits.BinTableHDU.from_columns([acol])
        assert ahdu.data.tostring().decode('raw-unicode-escape') == s
        ahdu.writeto(self.temp('newtable.fits'))
        with fits.open(self.temp('newtable.fits')) as hdul:
            assert hdul[1].data.tostring().decode('raw-unicode-escape') == s
            assert (hdul[1].data['MEMNAME'] == a).all()
        del hdul

        ahdu = fits.TableHDU.from_columns([acol])
        with ignore_warnings():
            ahdu.writeto(self.temp('newtable.fits'), clobber=True)

        with fits.open(self.temp('newtable.fits')) as hdul:
            assert (hdul[1].data.tostring().decode('raw-unicode-escape') ==
                    s.replace('\x00', ' '))
>           assert (hdul[1].data['MEMNAME'] == a).all()
E           assert <built-in method all of numpy.ndarray object at 0x1162e02b0>()
E            +  where <built-in method all of numpy.ndarray object at 0x1162e02b0> = array(['img1      ', 'img2      ', 'img3a     ', 'p         '], \n      dtype='|S10') == ['img1', 'img2', 'img3a', 'p'].all

astropy/io/fits/tests/test_table.py:1594: AssertionError
____________________________________________________________________ TestTableFunctions.test_string_array_round_trip ____________________________________________________________________

self = <astropy.io.fits.tests.test_table.TestTableFunctions object at 0x116f3d2d0>

    def test_string_array_round_trip(self):
        """Regression test for https://aeon.stsci.edu/ssb/trac/pyfits/ticket/201"""

        data = [['abc', 'def', 'ghi'],
                ['jkl', 'mno', 'pqr'],
                ['stu', 'vwx', 'yz ']]

        recarr = np.rec.array([(data,), (data,)], formats=['(3,3)S3'])

        t = fits.BinTableHDU(data=recarr)
        t.writeto(self.temp('test.fits'))

        with fits.open(self.temp('test.fits')) as h:
            assert 'TDIM1' in h[1].header
            assert h[1].header['TDIM1'] == '(3,3,3)'
            assert len(h[1].data) == 2
            assert len(h[1].data[0]) == 1
>           assert (h[1].data.field(0)[0] ==
                    recarr.field(0)[0].decode('ascii')).all()
E           AttributeError: 'numpy.ndarray' object has no attribute 'decode'

astropy/io/fits/tests/test_table.py:1687: AttributeError
___________________________________________________________________ TestTableFunctions.test_new_table_with_nd_column ____________________________________________________________________

self = <astropy.io.fits.tests.test_table.TestTableFunctions object at 0x116f50990>

    def test_new_table_with_nd_column(self):
        """Regression test for
            https://github.com/spacetelescope/PyFITS/issues/3
            """

        arra = np.array(['a', 'b'], dtype='|S1')
        arrb = np.array([['a', 'bc'], ['cd', 'e']], dtype='|S2')
        arrc = np.array([[[1, 2], [3, 4]], [[5, 6], [7, 8]]])

        cols = [
            fits.Column(name='str', format='1A', array=arra),
            fits.Column(name='strarray', format='4A', dim='(2,2)',
                        array=arrb),
            fits.Column(name='intarray', format='4I', dim='(2, 2)',
                        array=arrc)
        ]

        hdu = fits.BinTableHDU.from_columns(fits.ColDefs(cols))
        hdu.writeto(self.temp('test.fits'))

        with fits.open(self.temp('test.fits')) as h:
            # Need to force string arrays to byte arrays in order to compare
            # correctly on Python 3
>           assert (h[1].data['str'].encode('ascii') == arra).all()
E           AttributeError: 'numpy.ndarray' object has no attribute 'encode'

astropy/io/fits/tests/test_table.py:1727: AttributeError
_____________________________________________________________________ TestTableFunctions.test_dump_load_round_trip ______________________________________________________________________

self = <astropy.io.fits.tests.test_table.TestTableFunctions object at 0x1164ef390>

    def test_dump_load_round_trip(self):
        """
            A simple test of the dump/load methods; dump the data, column, and
            header files and try to reload the table from them.
            """

        hdul = fits.open(self.data('table.fits'))
        tbhdu = hdul[1]
        datafile = self.temp('data.txt')
        cdfile = self.temp('coldefs.txt')
        hfile = self.temp('header.txt')

        tbhdu.dump(datafile, cdfile, hfile)

        new_tbhdu = fits.BinTableHDU.load(datafile, cdfile, hfile)

>       assert comparerecords(tbhdu.data, new_tbhdu.data)
E       assert comparerecords(array([('NGC1001', 11.1), ('NGC1002', 12.3), ('NGC1003', 15.2)], \n      dtype=[('target', 'S20'), ('V_mag', '>f4')]).view(numpy.recarray), rec.array([('NGC1001             ', 11.1), ('NGC1002             ', 12.3),\n ('...            ', 15.2)], \n          dtype=[('target', '|S20'), ('V_mag', '<f4')]))
E        +  where array([('NGC1001', 11.1), ('NGC1002', 12.3), ('NGC1003', 15.2)], \n      dtype=[('target', 'S20'), ('V_mag', '>f4')]).view(numpy.recarray) = <astropy.io.fits.hdu.table.BinTableHDU object at 0x1164effd0>.data
E        +  and   rec.array([('NGC1001             ', 11.1), ('NGC1002             ', 12.3),\n ('...            ', 15.2)], \n          dtype=[('target', '|S20'), ('V_mag', '<f4')]) = <astropy.io.fits.hdu.table.BinTableHDU object at 0x1164efa50>.data

astropy/io/fits/tests/test_table.py:1845: AssertionError
------------------------------------------------------------------------------------ Captured stdout ------------------------------------------------------------------------------------
fielda:  ['NGC1001' 'NGC1002' 'NGC1003']
fieldb:  ['NGC1001             ' 'NGC1002             ' 'NGC1003             ']
field 0 differs
_______________________________________________________________________ TestTableFunctions.test_load_guess_format _______________________________________________________________________

self = <astropy.io.fits.tests.test_table.TestTableFunctions object at 0x1164ef090>

    def test_load_guess_format(self):
        """
            Tests loading a table dump with no supplied coldefs or header, so that
            the table format has to be guessed at.  There is of course no exact
            science to this; the table that's produced simply uses sensible guesses
            for that format.  Ideally this should never have to be used.
            """

        # Create a table containing a variety of data types.
        a0 = np.array([False, True, False], dtype=np.bool)
        c0 = fits.Column(name='c0', format='L', array=a0)

        # Format X currently not supported by the format
        # a1 = np.array([[0], [1], [0]], dtype=np.uint8)
        # c1 = fits.Column(name='c1', format='X', array=a1)

        a2 = np.array([1, 128, 255], dtype=np.uint8)
        c2 = fits.Column(name='c2', format='B', array=a2)
        a3 = np.array([-30000, 1, 256], dtype=np.int16)
        c3 = fits.Column(name='c3', format='I', array=a3)
        a4 = np.array([-123123123, 1234, 123123123], dtype=np.int32)
        c4 = fits.Column(name='c4', format='J', array=a4)
        a5 = np.array(['a', 'abc', 'ab'])
        c5 = fits.Column(name='c5', format='A3', array=a5)
        a6 = np.array([1.1, 2.2, 3.3], dtype=np.float64)
        c6 = fits.Column(name='c6', format='D', array=a6)
        a7 = np.array([1.1 + 2.2j, 3.3 + 4.4j, 5.5 + 6.6j],
                      dtype=np.complex128)
        c7 = fits.Column(name='c7', format='M', array=a7)
        a8 = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]], dtype=np.int32)
        c8 = fits.Column(name='c8', format='PJ()', array=a8)

        tbhdu = fits.BinTableHDU.from_columns([c0, c2, c3, c4, c5, c6, c7, c8])

        datafile = self.temp('data.txt')
        tbhdu.dump(datafile)

        new_tbhdu = fits.BinTableHDU.load(datafile)

        # In this particular case the record data at least should be equivalent
>       assert comparerecords(tbhdu.data, new_tbhdu.data)
E       assert comparerecords(rec.array([ (False, 1, -30000, -123123123, 'a', 1.1000000000000001, (1.1000000...4', '<i4'), ('c5', '|S3'), ('c6', '<f8'), ('c7', '<c16'), ('c8', '<i4', (2,))]), rec.array([ (False, 1, -30000, -123123123, 'a  ', 1.1000000000000001, (1.10000...3', '<i4'), ('f4', '|S3'), ('f5', '<f8'), ('f6', '<c16'), ('f7', '|u1', (3,))]))
E        +  where rec.array([ (False, 1, -30000, -123123123, 'a', 1.1000000000000001, (1.1000000...4', '<i4'), ('c5', '|S3'), ('c6', '<f8'), ('c7', '<c16'), ('c8', '<i4', (2,))]) = <astropy.io.fits.hdu.table.BinTableHDU object at 0x1169ac890>.data
E        +  and   rec.array([ (False, 1, -30000, -123123123, 'a  ', 1.1000000000000001, (1.10000...3', '<i4'), ('f4', '|S3'), ('f5', '<f8'), ('f6', '<c16'), ('f7', '|u1', (3,))]) = <astropy.io.fits.hdu.table.BinTableHDU object at 0x1171d0590>.data

astropy/io/fits/tests/test_table.py:1911: AssertionError
------------------------------------------------------------------------------------ Captured stdout ------------------------------------------------------------------------------------
fielda:  ['a' 'abc' 'ab']
fieldb:  ['a  ' 'abc' 'ab ']
field 4 differs
===================================================== 13 failed, 8501 passed, 123 skipped, 43 xfailed, 1 xpassed in 329.06 seconds ======================================================
@embray

This comment has been minimized.

Member

embray commented Feb 2, 2015

Surely the problem started sometime before that commit though, since it was just a documentation update.

@embray

This comment has been minimized.

Member

embray commented Feb 2, 2015

Almost certainly caused by numpy/numpy@3cd9e73 Looks like they've stopped returning string fields as chararray objects. Which in principle is a good change, but it's going to be a bit of a problem for FITS support since chararray is needed to properly support treatment of trailing whitespace.

My future goal was to add a new string type to Numpy that inherently ignores trailing whitespace. Not sure when I'll ever be able to get to that. In the meantime this will need a workaround eventually.

@embray embray added the io.fits label Feb 2, 2015

@embray embray modified the milestones: Future, v1.0.0 Feb 2, 2015

@embray

This comment has been minimized.

Member

embray commented Feb 2, 2015

Since this is in Numpy-dev there's no reason it needs to be fixed right away.

@mhvk

This comment has been minimized.

Contributor

mhvk commented Mar 31, 2015

This problem (and what look like related ones) also occurs under python3; updating the title of this issue.

@mhvk mhvk changed the title from 1.0rc1: test failures in io.fits with Python 2.x and Numpy dev to 1.0rc1: test failures in io.fits with Numpy dev Mar 31, 2015

@mhvk

This comment has been minimized.

Contributor

mhvk commented Mar 31, 2015

Since this is in Numpy-dev there's no reason it needs to be fixed right away.

@embray - this remains true, though for my work on __numpy_ufunc__ it would be quite helpful to not have to exclude specific parts of astropy from testing. I did have a quick look at io.fits, but realised that it is out of my league...

@embray

This comment has been minimized.

Member

embray commented Mar 31, 2015

The main issue, at least that I'm aware of, is that character fields in FITS_rec need to be forcibly returned as chararray objects (this used to come for free from the underlying recarray class, but they removed that, probably in an effort to fully remove chararray).

In the long-term the functionality of chararray will have to be replaced outright (the key functionality in question being how it ignores trailing whitespace in strings). In the medium-term it can still be used though.

dhomeier pushed a commit to dhomeier/astropy that referenced this pull request Mar 31, 2015

@mhvk

This comment has been minimized.

Contributor

mhvk commented May 22, 2015

@embray - as numpy 1.10 is now approaching, I'm getting back to trying to get __numpy_ufunc__ to work well. But obviously before doing that, current code should support numpy 1.10 and for that we still need to fix the bugs in io.fits. I tried briefly (well, took me several hours) to see whether one could easily fix the failures, but found that is prevented by recarray being used a little much on its own (not just in FITS_rec; hence just overwriting FITS_rec.__getattribute__ did not quite suffice).

However, I did confirm that if one simply uses the numpy 1.9.2 version of recarray, all tests pass under numpy-dev; see https://github.com/astropy/astropy/compare/master...mhvk:io-fits-old-recarray?expand=1

@embray

This comment has been minimized.

Member

embray commented May 22, 2015

Is this still just the issue of recarray not returning chararray objects for string fields?

@mhvk

This comment has been minimized.

Contributor

mhvk commented May 22, 2015

@embray - that certainly seems to be the case, as that is the only significant different between recarray in numpy 1.9.2 and 1.10-dev. Though a quick try with importing the new recarray and just overwriting __getattribute__ with the old version failed (not quite sure why; didn't have time to investigate in more detail).

@embray

This comment has been minimized.

Member

embray commented May 26, 2015

I'll see what I can do

@embray

This comment has been minimized.

Member

embray commented May 26, 2015

Looking into this uncovered another problem, not specific to any Numpy version but still related, that if one creates a binary table HDU from an array containing unicode fields, the resulting FITS file uses 4x the number of needed bytes for those columns, and just fills the extra space with zeros.

I'm probably going to make a change such that any data passed in to create a BinTableHDU is copied/converted if it is not already strictly FITS-formatted data.

@embray

This comment has been minimized.

Member

embray commented May 26, 2015

I've almost got this fixed, but there's actually a bug in Numpy now (due to related changes in Numpy) that is holding this up. I'm looking into the bug and seeing if I can't fix it.

@mhvk

This comment has been minimized.

Contributor

mhvk commented May 26, 2015

Great! Hope it works out; otherwise, even the partial solution may be good to have.

@embray

This comment has been minimized.

Member

embray commented May 27, 2015

See numpy/numpy#5921, which is what held things up a bit. Pretty sure I can work around it though.

@mhvk

This comment has been minimized.

Contributor

mhvk commented May 27, 2015

Nice find. Hopefully, they'll react quickly so there will be no need to work around it.

@embray

This comment has been minimized.

Member

embray commented May 27, 2015

Oh, that's true. This only affects 1.10-dev, so if this can be merged before 1.10 final is released then there will be no need for a workaround to that issue. I'll hold off then (I do have some other fixes though related to the chararray issue that need to be committed to astropy).

@mhvk

This comment has been minimized.

Contributor

mhvk commented May 27, 2015

OK, makes sense. if there are parts that do not depend on the numpy fixes, I think it makes sense to get those in already. The long PRs are often hard to review!

@embray

This comment has been minimized.

Member

embray commented May 27, 2015

I think other than that one Numpy issue I have all the other tests fixed, except for the ones where the FITS_rec repr also needs to be fixed (might be a good excuse to improve it a bit in the process).

embray added some commits May 27, 2015

Fixes most of the tests that are broken with the current (as of this …
…commit message) numpy-dev. Some of the tests are also fixed by numpy/numpy#5921 it doesn't get merged before Numpy 1.10 is out a workaround will have to be developed.
Fix failing test due to change in np.recarray.__repr__. This should b…
…e seen as more of a temporary fix for now. I would like to improve FITS_rec.__repr__ in general (bring it closer to Table.__repr__), but I'm starting with the basic fix for backporting purposes.
@embray

This comment has been minimized.

Member

embray commented May 27, 2015

The attached PR fixes all the issues except for the ones fixed by numpy/numpy#5921.
If some form of that doesn't get merged soon I'll attach an additional fix to work around.

@mhvk

This comment has been minimized.

Contributor

mhvk commented May 28, 2015

@embray - the changes look good. Does the numpy PR invalidate any of the changes here? If not, my tendency would be to merge this as is, and raise a new issue for the remaining test failure, just so we don't forget that it needs a workaround if not resolved by the numpy PR.

@embray

This comment has been minimized.

Member

embray commented May 29, 2015

Ok. No, other Numpy changes don't impact these ones.

@mhvk

This comment has been minimized.

Contributor

mhvk commented May 29, 2015

Ok. No, other Numpy changes don't impact these ones.

If so, since everything would be fixed with the further numpy PRs, I would suggest merging this and just having a quick issue to remind us that the numpy PRs are outstanding.

@embray

This comment has been minimized.

Member

embray commented May 29, 2015

Ok. I'll slip this in to v1.0.3 while there's still time.

@embray embray modified the milestones: v1.0.3, Future May 29, 2015

embray added a commit that referenced this pull request May 29, 2015

Merge pull request #3419 from embray/fits/issue-3419
1.0rc1: test failures in io.fits with Numpy dev

@embray embray merged commit e3c9398 into astropy:master May 29, 2015

@embray embray deleted the embray:fits/issue-3419 branch May 29, 2015

embray added a commit that referenced this pull request May 29, 2015

Merge pull request #3419 from embray/fits/issue-3419
1.0rc1: test failures in io.fits with Numpy dev

dhomeier added a commit to dhomeier/astropy that referenced this pull request Aug 11, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment