Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

math/big: Float.Float64() may not round correctly for values near math.SmallestNonzeroFloat64 #44058

Open
griesemer opened this issue Feb 2, 2021 · 3 comments
Assignees
Milestone

Comments

@griesemer
Copy link
Contributor

@griesemer griesemer commented Feb 2, 2021

The compiler constant expression

math.SmallestNonzeroFloat64 / 2

gets printed as 5e-324 (program), but the exact computation of math.SmallestNonzeroFloat64 / 2 using math/big arithmetic followed by rounding results in 0 (program).

There's either an error with the rounding or constant arithmetic somewhere, or a mistake in my assumptions.

This affects a test case in types2/testdata/const1.src.

@griesemer griesemer added this to the Go1.17 milestone Feb 2, 2021
@griesemer griesemer self-assigned this Feb 2, 2021
@dreamerjackson
Copy link
Contributor

@dreamerjackson dreamerjackson commented Feb 2, 2021

i think it's a mistake, in golang compile, the math.SmallestNonzeroFloat64 / 2 will use big.Float to calculate multi-precision floating, and when transform to float64 the precision will loss, so the print will undefined.

and your example use big.Rat, it's different

@dreamerjackson
Copy link
Contributor

@dreamerjackson dreamerjackson commented Feb 2, 2021

math.SmallestNonzeroFloat64 / 2 is a special example will rounding to math.SmallestNonzeroFloat64, the logic is here。change the types2/testdata/const1.src

func (x *Float) Float64() (float64, Accuracy) {
		if e < emin {
			// recompute precision
			p = mbits + 1 - emin + int(e)
			if p < 0 /* m <= 0.25 */ || p == 0 && x.mant.sticky(uint(len(x.mant))*_W-1) == 0 /* m == 0.5 */ {
				// underflow to ±0
				if x.neg {
					var z float64
					return -z, Above
				}
				return 0.0, Below
			}
			// otherwise, round up
			// We handle p == 0 explicitly because it's easy and because
			// Float.round doesn't support rounding to 0 bits of precision.
			if p == 0 {
				if x.neg {
					return -math.SmallestNonzeroFloat64, Below
				}
				return math.SmallestNonzeroFloat64, Above
			}
		}
}
@gopherbot
Copy link

@gopherbot gopherbot commented Feb 2, 2021

Change https://golang.org/cl/288672 mentions this issue: math/big: fix SmallestNonzeroFloat64 const tesedata data

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

3 participants