math/big: printing a MaxPrec big.Float panics if it was set using a hex format string

[kstenerud]

What version of Go are you using (go version)?

play.golang.org

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

play.golang.org

What did you do?

https://play.golang.org/p/Ou3piPGDOAF

package main

import (
	"fmt"
	"math/big"
)

func main() {
	flt := &big.Float{}
	flt.SetPrec(big.MaxPrec) // Remove this and it works
	flt.SetString("0x1.5")
	fmt.Printf("%v\n", flt)
}

What did you expect to see?

1.3125

What did you see instead?

%!v(PANIC=Format method: underflow)

[ALTree]

Yeah, could probably be handled better? Note that:

MaxPrec = math.MaxUint32 // largest (theoretically) supported precision; likely memory-limited

In practise I'm not even sure MaxPrec is a viable option, and looking at the math/big code seems to be a little under-tested. It's not just printing hex, flt.SetString("1.1") also fails with an OOM on the playground. I looks like a few math/big functions do not handle prec being set to MaxPrec.

[kstenerud]

Mainly I just want to be able to set a big.Float from a hex string and not have it round the value. It's fine to have a "sane" maximum value, but like most people I'm enforcing a maximum number of digits when converting from string, which doesn't translate well to a precision value.

I don't actually need MaxPrec; I just want to be confident that my converted values are accurate to the number of digits entered.

[ALTree]

So basically you need the Parse equivalent of what the g flag makes Text do:

A negative precision selects the smallest number of decimal digits necessary to identify the value x uniquely using x.Prec() mantissa bits.

For example you could pass -1 as the precision in big.ParseFloat and it would return a *Float with its precision set to the minimum necessary to represent the string. (Note that ParseFloat takes an uint prec so we can't overload a prec of -1 to mean that, we'll need something different).

As far as I know, this is not possible with the current math/big.Float API.

I don't think it could support any base (if you pass a decimal, for example, it could be periodic in the base math/big uses, so there wouldn't be a "smallest prec that can accurately represent the value").

In base 16 this can be done, but you could also do the computation yourself (from number of digits to precision), and then call SetPrec with that precision.