TST: ascii.write(outtab, format='mrt') doctest failed in remote data job

[pllim]

Can the people involved with this new MRT format please have a look and submit a fix? Thanks!

_____________________________ [doctest] write.rst ______________________________
397 ``sys.stdout`` instead of a file::
398 
399   >>> table['error'] = [1e4, 450] * u.Jy  # Error in the Flux values.
400   >>> outtab = table.copy()  # So that changes don't affect the original table.
401   >>> outtab.rename_column('error', 'e_Flux')
402   >>> # re-order so that related columns are placed next to eachother.
403   >>> outtab = outtab['Name', 'Obs', 'coord', 'Cadence', 'nH', 'magnitude',
404   ...                 'Temperature', 'Flux', 'e_Flux']  # doctest: +REMOTE_DATA
405 
406   >>> ascii.write(outtab, format='mrt')  # doctest: +REMOTE_DATA
Differences (unified diff with -expected +actual):
    @@ -7,17 +7,17 @@
      Bytes Format Units  Label     Explanations
     --------------------------------------------------------------------------------
    - 1- 11  A11     ---    Name        Description of Name
    -13- 18  F6.1    ---    Obs         [2019.0/2019.0] Time of Observation
    -20- 22  I3      s      Cadence     [100] Description of Cadence
    -24- 29  F6.4    10+22  nH          [0.01/0.03] Description of nH
    -31- 35  E5.1    mag    magnitude   [0.0/3981.08] Description of magnitude
    -37- 47  E11.6   keV    Temperature [0.0/0.01] Description of Temperature
    -49- 53  F5.3   10+12Jy Flux        ? Description of Flux
    -55- 61  F7.1    Jy     e_Flux      [450.0/10000.0] Description of e_Flux
    -63- 78  F16.12  deg    ELON        Ecliptic Longitude (geocentrictrueecliptic)
    -80- 95  F16.12  deg    ELAT        Ecliptic Latitude (geocentrictrueecliptic)
    +  1- 11  A11     ---    Name        Description of Name                        
    + 13- 18  F6.1    ---    Obs         [2019.0/2019.0] Time of Observation        
    + 20- 22  I3      s      Cadence     [100] Description of Cadence               
    + 24- 29  F6.4    10+22  nH          [0.01/0.03] Description of nH              
    + 31- 35  E5.1    mag    magnitude   [0.0/3981.08] Description of magnitude     
    + 37- 47  E11.6   keV    Temperature [0.0/0.01] Description of Temperature      
    + 49- 53  F5.3   10+12Jy Flux        ? Description of Flux                      
    + 55- 61  F7.1    Jy     e_Flux      [450.0/10000.0] Description of e_Flux      
    + 63- 78  F16.12  deg    ELON        Ecliptic Longitude (geocentrictrueecliptic)
    + 80- 95  F16.12  deg    ELAT        Ecliptic Latitude (geocentrictrueecliptic) 
     --------------------------------------------------------------------------------
     Notes:
     --------------------------------------------------------------------------------
    -ASASSN-15lh 2019.0 100 0.0250 1e-10 2.87819e-09       10000.0 306.224208650096 -45.621789850825
    -ASASSN-14li 2019.0 100 0.0188 4e+03 2.55935e-08 2.044   450.0 183.754980099243  21.051410763027
    +ASASSN-15lh 2019.0 100 0.0250 1e-10 2.87819e-09       10000.0 306.224208728013 -45.621789755268
    +ASASSN-14li 2019.0 100 0.0188 4e+03 2.55935e-08 2.044   450.0 183.754980100354  21.051410762137

.../docs/io/ascii/write.rst:406: DocTestFailure
=========================== short test summary info ============================

[dhomeier]

Hmm, the last two lines obviously could do with a +FLOAT_CMP (have always been wondering there were not more failures in those tests), but as to why the writer would suddenly switch to 3d format for the Bytes columns I am at a loss!

[dhomeier]

switch to 3d format for the [1st] Bytes columns I am at a loss!

Or for that matter, why it was already expected in the 2nd column, but not in the 1st – when I am writing that table with 5.0, I am getting

Title:
Authors:
Table:
================================================================================
Byte-by-byte Description of file: table.dat
--------------------------------------------------------------------------------
 Bytes Format Units  Label     Explanations
--------------------------------------------------------------------------------
 1-11  A11     ---    Name        Description of Name                        
13-18  F6.1    ---    Obs         [2019.0/2019.0] Time of Observation        
20-22  I3      s      Cadence     [100] Description of Cadence               
24-29  F6.4    10+22  nH          [0.01/0.03] Description of nH              
31-35  E5.1    mag    magnitude   [0.0/4000.0] Description of magnitude      
37-47  E11.6   keV    Temperature [0.0/0.01] Description of Temperature      
49-53  F5.3   10+12Jy Flux        ? Description of Flux                      
55-61  F7.1    Jy     e_Flux      [450.0/10000.0] Description of e_Flux      
63-78  F16.12  deg    ELON        Ecliptic Longitude (geocentrictrueecliptic)
80-95  F16.12  deg    ELAT        Ecliptic Latitude (geocentrictrueecliptic) 
--------------------------------------------------------------------------------
Notes:
--------------------------------------------------------------------------------
ASASSN-15lh 2019.0 100 0.0250 1e-10 2.87819e-09       10000.0 306.224208650096 -45.621789850825
ASASSN-14li 2019.0 100 0.0188 4e+03 2.55935e-08 2.044   450.0 183.754980099243  21.051410763027

which is actually what should be expected, since the maximum values in both columns are double-digit.

[dhomeier]

Looking at the code it becomes even more incomprehensible how the two columns could have different widths at all, since they are identically formatted as singlebfmt

astropy/astropy/io/ascii/mrt.py

Lines 239 to 254 in e97ebc7

	for i, col in enumerate(self.cols):
	col.width = max([len(vals[i]) for vals in vals_list])
	if self.start_line is not None:
	col.width = max(col.width, len(col.info.name))
	widths = [col.width for col in self.cols]
	startb = 1 # Byte count starts at 1.
	# Set default width of the Bytes count column of the Byte-By-Byte table.
	# This ``byte_count_width`` value helps align byte counts with respect
	# to the hyphen using a format string.
	byte_count_width = len(str(sum(widths) + len(self.cols) - 1))
	# Format string for Start Byte and End Byte
	singlebfmt = "{:" + str(byte_count_width) + "d}"
	fmtb = singlebfmt + "-" + singlebfmt

But it looks like the reason for the inconsistent behaviour is the use of len(vals[i]) as width, where I suspect vals[i] is the (machine-dependent) default string representation of the columns in self.data. Actually this should be col.formatted_width which is however only determined later, when already in the header-writing iteration through the columns.

[pllim]

So, there is no quick fix to make the test pass again?

[dhomeier]

Not really quick I am afraid (other than marking that example +SKIP as well).
One could probably store the content part (everything from Format on) of the header lines in a list and then run a second iteration to prepend the Bytes columns.