Assigning new data to an existing HDU won't update scaling consistently

[rcardenes]

When creating a brand new ImageHDU, this:

im = ImageHDU(data=data, header=existing_header)

will deal properly with the header scaling, adding or subtracting BZERO/BSCALE to fit the data type, no matter what the data type is, or whether the original header has scaling information on it or not. If we set the data after creation, the behavior is not consistent. Let's show this.

>>> uint8 = np.array([[1,2,3,4],[4,3,2,1]], dtype='uint8')
>>> uint64 = np.array([[1,2,3,4],[4,3,2,1]], dtype='uint64')

Now, let's create a new image with a data type that doesn't require scaling, and replace the data with the unsigned 64 bit array:

>>> im = ImageHDU(uint8)
>>> im.header
XTENSION= 'IMAGE   '           / Image extension                                
BITPIX  =                   8 / array data type                                
NAXIS   =                    2 / number of array dimensions                     
NAXIS1  =                    4                                                  
NAXIS2  =                    2                                                  
PCOUNT  =                    0 / number of parameters                           
GCOUNT  =                    1 / number of groups                               
>>> im.data = uint64
>>> im.header
XTENSION= 'IMAGE   '           / Image extension                                
BITPIX  =                   64 / array data type                                
NAXIS   =                    2 / number of array dimensions                     
NAXIS1  =                    4                                                  
NAXIS2  =                    2                                                  
PCOUNT  =                    0 / number of parameters                           
GCOUNT  =                    1 / number of groups                               
BSCALE  =                    1                                                  
BZERO   =  9223372036854775808                                                  

It behaves as expected. Now:

>>> im = ImageHDU(uint64)
>>> im.header
XTENSION= 'IMAGE   '           / Image extension                             
BITPIX  =                   64 / array data type                                
NAXIS   =                    2 / number of array dimensions                  
NAXIS1  =                    4                                               
NAXIS2  =                    2                                               
PCOUNT  =                    0 / number of parameters                        
GCOUNT  =                    1 / number of groups                            
BSCALE  =                    1                                               
BZERO   =  9223372036854775808                                               
>>> im.data = uint8
>>> im.header
XTENSION= 'IMAGE   '           / Image extension                             
BITPIX  =                    8 / array data type                                
NAXIS   =                    2 / number of array dimensions                  
NAXIS1  =                    4                                               
NAXIS2  =                    2                                               
PCOUNT  =                    0 / number of parameters                        
GCOUNT  =                    1 / number of groups                            
BSCALE  =                    1                                               
BZERO   =  9223372036854775808                                               

As we can see, scaling info is preserved, when it shouldn't. This works properly if the new array is of 16/32/64 integer (signed or unsigned), but fails with 8 bit and floating point data.

[MSeifert04]

Thanks for reporting this issue.

I have one question: bscale and bzero are only needed when reading a file - Are these values preserved or updated when you write your im to a file and read it back?

Also there have been several edits to bscale and bzero, which version of astropy are you using?

[rcardenes]

We're writing this back to disk, and the headers are preserved when writing, so when reading the data is screwed up. In principle I only discovered this because for some reason I can't really remember (sigh), I needed to create an image like this:

im = ImageHDU(data=DELAYED, header=the_header)
im.data = data

I'm not doing this any longer, so it shouldn't be an issue for us, but I still think it was worth reporting.

I've observed the bug both in AstroPy 1.2.1 and in 1.3.dev17297

[saimn]

It looks like this case was excluded on purpose in #5053, in particular #5053 (comment): do you remember why @mwcraig ?
From what I can understand it seems that the goal is to be able to know that the original data was unsigned, once it has been scaled.

If I modify the code to handle the case above, allowing the removal of BSCALE/BZERO when the new data is unsigned, then the test added in #5053 fails, as well as test_uint_header_consistency from #2305.