I/O module: Meaning of %.6 varies depending on what type it is applied to

[lydia-duncan]

The number after the decimal point in formatted string specification varies depending on the type of the argument it is applied to. For integers, it means "insert a decimal point and pad to the specified number of zeroes", while for reals, we only print out the number of decimals that were already provided. This is especially confusing when the format string written doesn't appear to have changed otherwise, such as when using the generic n to indicate the type required:

use IO.FormattedIO;

writef("%.6n\n", 35);    // prints `35.000000`
writef("%.6n\n", 2.13);  // prints `2.13`

Such differences are reflected when explicitly specifying the type:

use IO.FormattedIO;

writef("%.6i\n", 35);    // prints `35.000000`
writef("%.6r\n", 2.13);  // prints `2.13`

While it may make sense in an individual type situation, in the larger picture it seems confusing and could cause problems when copying formatting lines for adjustment later in your program.

Should we change this behavior? How?

[bradcray]

This is the kind of question that makes me want to ask "What do C and Python do?" (where I suspect that C has it easier because AFAIK, it doesn't support type-neutral format specifiers like Chapel's %n; I'm less familiar with what the situation is in Python).

[lydia-duncan]

From some brief experimentation, it looks like Python:

• always pads to the specified amount when used with the explicit float specifier:

>>> x = 3.2637
>>> y = 2
>>> print(f'{x:.3f}')
3.264
>>> print(f'{y:.3f}')
2.000
>>> z = 4.1
>>> print(f'{z:.3f}')
4.100

• doesn't allow precision to be applied to integers without decimals, even using the generic number specifier (the specification says n functions like d, the integer specifier)

>>> y = 2
>>> print(f'{y:.3d}')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Precision not allowed in integer format specifier
>>> print(f'{y:.3n}')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Precision not allowed in integer format specifier

• Treats .3 as referring to the number of digits in the number (not decimal places) and will not pad smaller decimal numbers out to fill the space (this is different from the behavior from using f)

>>> x = 3.2637
>>> z = 4.1
>>> print(f'{z:.3n}')
4.1
>>> print(f'{x:.3n}')
3.26

[lydia-duncan]

• As Brad said above, C does not allow general type specifiers. In contrast to Python (which doesn't support it on integers, see above) and us, it pads integers with zeroes before the number. But its behavior with floating point numbers matches Python.

#include <stdio.h>

int main() {
  printf("%.3i\n", 2); // prints `002`
  printf("%.3f\n", 3.2637); // prints `3.264`
  printf("%.3f\n", 4.1); // prints `4.100`
}

• Rust doesn't seem to allow you to specify the type as part of the format string, but will vary what it shows based on the type passed. Integers ignore precision specifiers, floating point numbers will use it to specify the number of digits after the decimal point (so will match Python and C).
• Julia just provides a wrapper for C's printf so far as I can tell
• Swift also does not allow general type specifiers and provides the same interface as C

[lydia-duncan]

Summary for the up-coming ad-hoc sub-team discussion:

What is the API being presented?

The number after the decimal point in formatted string specification varies depending on the type of the argument it is applied to. For integers, it means "insert a decimal point and pad to the specified number of zeroes", while for reals, we only print out the number of decimals that were already provided. This is especially confusing when the format string written doesn't appear to have changed otherwise, such as when using the generic n to indicate the type required:

use IO.FormattedIO;

writef("%.6n\n", 35);    // prints `35.000000`
writef("%.6n\n", 2.13);  // prints `2.13`

Such differences are reflected when explicitly specifying the type:

use IO.FormattedIO;

writef("%.6i\n", 35);    // prints `35.000000`
writef("%.6r\n", 2.13);  // prints `2.13`

While it may make sense in an individual type situation, in the larger picture it seems confusing and could cause problems when copying formatting lines for adjustment later in your program.

How is it intended to be used?

For reals and complexes it is used to truncate the number if it has more significant digits than would otherwise fit. Interestingly, the docs for integer format specifiers don't explicitly mention it as something that can be done, but Chapel accepts it today and it pads the number so it fills a certain width.

How is it being used in Arkouda and CHAMPS?

In Arkouda, there are 42 uses of this format modifier. All of them are with %r, there are no uses with %n, %i or %t.

Uses in Arkouda:

• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/MultiTypeSymbolTable.chpl#L479
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/ServerDaemon.chpl#L597
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/ServerDaemon.chpl#L676
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/ServerDaemon.chpl#L697
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/ServerDaemon.chpl#L711
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/ServerDaemon.chpl#L597
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/ReductionMsg.chpl#L71
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/ReductionMsg.chpl#L131
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/ReductionMsg.chpl#L181
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/ReductionMsg.chpl#L185
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/ReductionMsg.chpl#L189
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/ReductionMsg.chpl#L193
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/Cast.chpl#L70
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L27
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L30
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L33
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L36
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L57
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L60
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L63
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L66
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L87
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L90
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L93
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L96
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L119
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L124
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L129
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L134
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L139
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L144
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L149
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L154
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L159
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L164
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L169
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L174
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L179
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L184
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L189
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L194
• https://github.com/Bears-R-Us/arkouda/blob/af0570780bc52c47b2633accc646606b898b3544/src/StatsMsg.chpl#L212

In CHAMPS, there are 300+ uses, all of which are with %r. There are no uses with %n, %t or %i.

2. What's the history of the feature, if it already exists?

It was added with readf/writef support in 2013 and has mostly been unchanged since then.

3. What's the precedent in other languages, if they support it?

(see above)

4. Are there known Github issues with the feature?

• Formats are mentioned generally here, but not this specific aspect of them: Improve Channel Formats #7846

5. Are there features it is related to? What impact would changing or adding this feature have on those other features?

• It's related to the outcome of the %t discussion: I/O module: should we change or deprecate %t for readf/writef? #19906

6. How do you propose to solve the problem?

• Maintain the status quo (though we probably should document the behavior with ints)

  • Pros:
    • less effort
  • Cons:
    • Inconsistent across types, which could be confusing
    • Not consistent with other languages

• Modify our behavior so that the int version matches C/Julia/Swift (pad with zeroes before the number)

  • Pros:
    • Consistent with C/Julia/Swift and IEEE standard
    • Integers printed with this will be easily recognizable as integers when being read back
  • Cons:
    • Behavior will be inconsistent across types, which could be confusing

• Drop support for %.<num> with integers (arguably this could be categorized as a bug fix since we don't document it for ints).

  • Pros:
    • Matches Python
    • Arkouda and CHAMPS don't use it, arguably it is not useful to users
    • No documentation update
    • We already have some types that don't support it, so there's some precedent within our own implementation.
  • Cons:
    • Doesn't match C/Julia/Swift
    • Using %.<num> with %n and %t will still vary depending on what is sent (it's just sometimes now it'll be an error instead)

I favor dropping support for %.<num> with integers. I'd prefer not to maintain the status quo and would be fine with modifying it to match C/Julia/Swift

[lydia-duncan]

In our ad-hoc sub-team meeting today, we decided to:

• drop support for %.6i.
  • Jade: writing an integer with decimal points seems weird
• adjust %.6r to match C/Julia/Swift (where %.6n for the same number will also adjust in the same way)
  • Jade: expect that when %.6 is provided, padding will occur.
  • Jeremiah: slight preference for leaving it alone but okay with changing it
• allow %.6n when an integer is passed, but treat it like %r was used.
  • Jade: having it break when going from more specific to less specific seems counterintuitive
  • Lydia: writef("%.6r", 1) -> writef("%.6n", 1) seems like it should behave the same and with no %.6i, people probably won't expect any different behavior for %.6 and an integer.

[jeremiah-corrado]

Resolved by #22924