Support byteswap

[Cadair]

I am trying to add Dask support to astropy.io.fits and because it's such a lovely file format FITS is big endian on disk, so before I can .store() the data to the file I need to byteswap the array.

What I have currently done which seems to work is:

        @delayed
        def _byteswap(arr):
            return np.array(arr).byteswap(True).newbyteorder("S")

        return dask.array.from_delayed(_byteswap(array), shape=array.shape,
                                       dtype=array.dtype.newbyteorder("S"))

which appears to work but is somewhat weird.

I was wondering what would be involved in adding .byteswap to Array this seems to suggest it wouldn't be that hard, but I wouldn't know where to start.

[mrocklin]

@Cadair it looks like this is an embarassingly parallel operation, in which case, your current workaround might be faster as ...

out = x.map_blocks(np.ndarray.byteswap, True).map_blocks(np.ndarray.newbyteorder, "S")

Or, if you wanted to keep things numpy agnostic, maybe swap out np.ndarray for M as follows

from dask.utils import M

out = x.map_blocks(M.byteswap, True).map_blocks(M.newbyteorder, "S")

See https://docs.dask.org/en/latest/best-practices.html#learn-techniques-for-customization

If it is this simple then this would probably make a fine contribution to the dask array module. You might search for similarly simple methoods, like clip to see examples of what needs to be touched.

[Cadair]

Thanks @mrocklin

I swapped out my implementation with what you suggested, and it didn't work right away. I haven't looked into it any more though, will try and debug later.

[TomAugspurger]

How are things going here @Cadair? Anything we can help with?