Floating point precision in fits header keywords

[hamogu]

Description

The precision of the formatting for floating point numbers in fits header keywords is chosen, such that some number look exactly as input (23 -> 23) but others appear different (99.9 -> 99.90000000000001 ). This is obviously due to the limited precision of floating point numbers, but could be avoided by choosing a different formatting when the floating point number is converted to a ASCII string.

(This was reported to me by email by a user, I'm posting it here with a minimal example and slightly reworded.)

Expected behavior

Float number in the header are formatted with a precision that matches (a) either their true precision or (b) the precision of the most common precision used. Personally, I suggest to format to just one less digit, as that seems to work for 64-bit floats which are by far the most common in Python.

Actual behavior

The precision of the formatting for floating point numbers in fits header keywords is chosen, such that some number look exactly as input (23 -> 23) but others appear different (99.9 -> 99.90000000000001 ).

Steps to Reproduce

from astropy.io import fits

# Just use table to quickly make a short fits file to experiment on
from astropy.table import Table
tab = Table({'a': [1,2]})
tab.write('text.fits')

fits.setval('text.fits', 'D_APDTI', value=-99.9)
fits.setval('text.fits', 'D_APDTI2', value=23)

Then, outside of Python (e.g. less text.fits), I see:

SIMPLE  =                    T / conforms to FITS standard                      
BITPIX  =                    8 / array data type                                
NAXIS   =                    0 / number of array dimensions                     
EXTEND  =                    T                                                  
D_APDTI =   -99.90000000000001                                                  
D_APDTI2=                   23                                                  
END                                                                                                                                                                           

System Details

maxOS-12.2.1-arm64-arm-64bit
Python 3.10.1 | packaged by conda-forge | (main, Dec 22, 2021, 01:39:07) [Clang 11.1.0]
Numpy 1.22.0
astropy 5.0

[DasVinch]

FYI, I am the original poster, huge thanks to @hamogu for taking the matter here.

[saimn]

Astropy currently uses a %.16G format (see also #5449), which I'm not sure is the best solution. I raised this issue a long time ago, but I don't know if Python's default formatting is clever enough so we could just rely on it instead of Astropy's current default.

[hamogu]

This is essentially a duplicate of #5449, so we can close this issue. I searched, but did not find that one.