math/big: Float.Float64() may not round correctly for values near math.SmallestNonzeroFloat64

[griesemer]

The compiler constant expression

math.SmallestNonzeroFloat64 / 2

gets printed as 5e-324 (program), but the exact computation of math.SmallestNonzeroFloat64 / 2 using math/big arithmetic followed by rounding results in 0 (program).

There's either an error with the rounding or constant arithmetic somewhere, or a mistake in my assumptions.

This affects a test case in types2/testdata/const1.src.

[dreamerjackson]

i think it's a mistake, in golang compile, the math.SmallestNonzeroFloat64 / 2 will use big.Float to calculate multi-precision floating， and when transform to float64 the precision will loss, so the print will undefined.

and your example use big.Rat, it's different

[dreamerjackson]

math.SmallestNonzeroFloat64 / 2 is a special example will rounding to math.SmallestNonzeroFloat64, the logic is here。change the types2/testdata/const1.src

func (x *Float) Float64() (float64, Accuracy) {
		if e < emin {
			// recompute precision
			p = mbits + 1 - emin + int(e)
			if p < 0 /* m <= 0.25 */ || p == 0 && x.mant.sticky(uint(len(x.mant))*_W-1) == 0 /* m == 0.5 */ {
				// underflow to ±0
				if x.neg {
					var z float64
					return -z, Above
				}
				return 0.0, Below
			}
			// otherwise, round up
			// We handle p == 0 explicitly because it's easy and because
			// Float.round doesn't support rounding to 0 bits of precision.
			if p == 0 {
				if x.neg {
					return -math.SmallestNonzeroFloat64, Below
				}
				return math.SmallestNonzeroFloat64, Above
			}
		}
}

[gopherbot]

Change https://golang.org/cl/288672 mentions this issue: math/big: fix SmallestNonzeroFloat64 const tesedata data