strconv: ParseFloat should accept 'p' notation for binary exponents

[kortschak]

See https://groups.google.com/d/topic/golang-dev/oIB-wBj3ufw/discussion.

The language specification never mentioned binary exponent float representation, but it was previously included in the gc implementation and it is included as a formatting option via strconv.AppendFloat with the 'b' fmt argument. However, it now lives on as a parsing option only in the compiler and test code in strconv.

The capacity to represent exact float values in a clear human-readable way is valuable in numeric code, for example here, where otherwise comments are required to explain the magic hex.

It is not clear how this should be included, since parsing a string is failable at runtime and these values are likely to nearly always be compile time constants.

/cc @griesemer

[griesemer]

Some comments:

1. The compiler is using the mechanism to write (export) and read (import) exported float constants, written using the p exponent, because it permits an easy and lossless representation of a float in decimal form (decimal mantissa, exponent to power of 2, but written in decimal form). Note that the need for this format is likely going away since a binary representation of the exported data is more compact, just as precise, and faster to read and write (I'm working on a respective change).

2. The strconv.AppendFloat representation of the 'b' format requires the bitsize of the argument (32 or 64). It simply interprets the mantissa as a large decimal, and then prints the exponent. For instance, a float64 0.0, using 'b' format, formats as: "0p-1074" which is somewhat odd as it requires understanding of the 64bit float format to explain how the result was obtained. Similarly, 1.0 is printed as 4503599627370496p-52, that is it is the float64 mantissa (53 bits) interpreted as a decimal, which is then printed (4503599627370496, same as 1<<52), followed by the exponent (http://play.golang.org/p/Rt0SIFzzHi). Again, it requires the mantissa size to explain the output. (0p0 and 1p0 would be just a valid, but be more expensive to derive - basically it's the same mantissa with trailing 0's removed and the exponent adjusted - a canonical form).

And some questions:

1. What are you proposing? (Is this a proposal?)
2. Are you arguing that this format should be acceptable syntax in the language?

[kortschak]

This is not a proposal, I wanted to sound things out first.

I don't think that it needs to be part of the language, the utility outside numerics is limited. At the most making strconv.ParseFloat handle it is what I am thinking.

[griesemer]

Having strconv.ParseFloat handle the format sounds reasonable to me. I think the next step would be to define the exact format (syntax). It's probably something like:

number = [sign] mantissa 'p' [sign] exponent.
sign = '+' | '-' .
mantissa = decimalDigit {decimalDigit}.
exponent = decimalDigit {decimalDigit}.

Questions:

• Should both 'p' and 'P' be permitted? Why/why not?
• Can the mantissa be hexadecimal? Why/why not?

[kortschak]

It seems to me that a single case for 'p' makes sense because it is a (marginally) simpler thing to look for instances (visually and mechanically) when there is only one thing to look for and that thing is visually distinct from a digit (in the decimal case). Mantissa optionally as hex makes moderate sense since a bit pattern may be what is being specified.

[griesemer]

Permitting only 'p' sounds good. Perhaps for a start, also leave away hexadecimal notation. Thus, a float in p notation is essentially a signed integer followed by a 'p' exponent.

Venture to send a change list? (strconv/atof.go)

[kortschak]

Yeah, I'll look into that. Just an initial observation though; it seems that fmt.Scan* handle binary exponent float representations judging by the test cases that exist (though also with dot rather than int-only mantissa). big.Float also handles these cases, but also includes hex input (including non-int).

Before I add to the variety, I'd like to get input on that.

[griesemer]

I haven't looked at fmt.Scan. Permitting a decimal point for a decimal mantissa is tricky. The point of the 'p' notation is 100% lossless conversion with a fast and simple algorithm. In general that's not true anymore once a decimal point is permitted.

big.Float uses a different format: the mantissa is represented by a hex number which corresponds to the bits after (to the right) of the "decimal" point - that is, that mantissa value m is 0.5 <= m < 1.0. It's essentially used for testing (and could possibly be changed).

Given a sign s (-1, +1), a mantissa m that is simply a decimal unsigned integer, and a binary exponent b, the floating point value x is x = s * m * 2**b . No further explanation needed.

There are design decisions to be made when printing using a binary exponent: The mantissa may be scaled arbitrarily. Currently, printing simply prints the float32/float64 mantissa bits like if they were int32 or int64 bit numbers (with appropriate exponent). This requires knowing the bit size of the type to reproduce. Another option would be to always print a canonical form; for instance such that the mantissa is the smallest possible value before requiring a decimal point. That is equivalent to having no trailing 0's in the mantissa (or the mantissa being odd, except for x == 0).

But for parsing it doesn't matter.

More generally: it seems that strconv conversion routines should parse numbers that it can print.

[kortschak]

Agreed. Just getting clarification.

[rsc]

@kortschak, regarding your initial comment:

The capacity to represent exact float values in a clear human-readable way is valuable in numeric code, for example here, where otherwise comments are required to explain the magic hex.

And the code says:

var (
    // dlamchE is the machine epsilon. For IEEE this is 2^-53.
    dlamchE = math.Float64frombits(0x3ca0000000000000)

    // dlamchP is 2 * eps
    dlamchP = math.Float64frombits(0x3cb0000000000000)

    // dlamchS is the "safe min", that is, the lowest number such that 1/sfmin does
    // not overflow. The Netlib code for calculating this number is not correct --
    // it overflows. Found by trial and error, it is equal to (1/math.MaxFloat64) * (1+ 6*eps)
    dlamchS = math.Float64frombits(0x4000000000001)

    ...
)

I want to make the point, unrelated to what we do in strconv, that this is unnecessary in Go. This kind of thing - specifying floating point constants in hexadecimal - is rampant in C because C compilers have historically been quite bad at reading floating point inputs. Using hex was the only way to guarantee the compiler arrived at the number you intended. But modern practice has improved, and Go gets this right. There are any number of ways you could write the above code using plain floating point constants, but the most direct is:

var (
    // dlamchE is the machine epsilon. For IEEE this is 2^-53.
    dlamchE = 1.1102230246251565e-16

    // dlamchP is 2 * eps
    dlamchP = 2.220446049250313e-16

    // dlamchS is the "safe min", that is, the lowest number such that 1/sfmin does
    // not overflow. The Netlib code for calculating this number is not correct --
    // it overflows. Found by trial and error, it is equal to (1/math.MaxFloat64) * (1+ 6*eps)
    dlamchS = 5.56268464626801e-309

    ...
)

This is guaranteed to have the same effect as the math.Float64frombits calls.

[rsc]

Postponing the strconv work.