spec: Float to int conversions underspecified

[cznic]

What version of Go are you using (go version)?

go version go1.8 linux/amd64

What operating system and processor architecture are you using (go env)?

GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/jnml"
GORACE=""
GOROOT="/home/jnml/go"
GOTOOLDIR="/home/jnml/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build750966123=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

What did you do?

If possible, provide a recipe for reproducing the error.

package main

import (
	"fmt"
	"math"
)

func main() {
	f := float32(math.MaxInt32 - 64)
	i := int32(f)
	fmt.Printf("%.10v, %v %#x\n", f, i, i)
	f = float32(math.MaxInt32 - 63)
	i = int32(f)
	fmt.Printf("%.10v, %v %#x\n", f, i, i)
}

playground

What did you expect to see?

2147483520, 2147483520 0x7fffff80
2147483648,  2147483647 0x7fffffff 

What did you see instead?

2147483520, 2147483520 0x7fffff80
2147483648, -2147483648 -0x80000000

Discussion

The only relevant specs part I've found says

For the conversion of non-constant numeric values, the following rules apply:

...
2. When converting a floating-point number to an integer, the fraction is discarded (truncation towards zero).
...

IOW, the outcome on overflow seems to be unspecified. However, changing the sign on overflow is surprising. Note that this is not a case of modular arithmetic. Bigger values of f keep the value of i unchanged

FWIW, gcc has also a cut-off, but w/o sign change: 20031003-1.c, although it has to be said that in C the overflow is UB, with some mostly related discussion for example here.

[cznic]

Sorry for the accidental close, phone touchscreens are the worst UI ever 😤

[cherrymui]

It is "specified" a few lines later:

In all non-constant conversions involving floating-point or complex values, if the result type cannot represent the value the conversion succeeds but the result value is implementation-dependent.

IIRC the sign on ARM64 and PPC64 doesn't change.
Related: issue #18762.

[cznic]

@cherrymui Thank you for the info, I missed that part of the specs because I was looking for 'float' and 'integer' together.

I'd say the issue now reduces to the problem of, IMHO, the overflow condition being a bit of a trap. Straight forward code like

if someFloat32value > float32(math.MaxInt32) {
        // handle overflow here
}

// We're safe to convert
someInt32value = int32(someFloat32value)

fails for someFloat32Value ∈ [math.MaxInt32 - 63, math.MaxIn32), moreover there's the sign surprise, even though, admittedly, it is an implementation-dependent value.

I think it might be nice to have the cut-off constants for all floating-point and complex types available in math.

[RLH]

Does ieee754 provide any guidance here? At the end of the day it is what the hardware will implement.

…

On Mon, Mar 6, 2017 at 5:18 AM, cznic ***@***.***> wrote: @cherrymui <https://github.com/cherrymui> Thank you for the info, I missed that part of the specs because I was looking for 'float' *and* 'integer' together. I'd say the issue now reduces to the problem of, IMHO, the overflow condition being a bit of a trap. Straight forward code like if someFloat32value > float32(math.MaxInt32) { // handle overflow here } // We're safe to convert someInt32value = int32(someFloat32value) fails for someFloat32Value ∈ [math.MaxInt32 - 63, math.MaxIn32), moreover there's the sign surprise, even though, admittedly, it *is* an *implementation-dependent* value. I think it might be nice to have the cut-off constants for all floating-point and complex types available in math. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#19405 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA7Wn0xFF6cTtl8vdwcj2kPchZ3OHO3mks5ri92TgaJpZM4MTYp4> .

[cznic]

The only thing I've found in IEEE754 is

5.8 Details of conversions from floating-point to integer formats
...
When a numeric operand would convert to an integer outside the range of the destination format, the invalid operation exception shall be signaled if this situation cannot otherwise be indicated.
...

[ianlancetaylor]

I don't think the problem with your suggested overflow check is the same as the issue with converting large values from float32 to int32. Your overflow check fails because converting math.MaxInt32 to float32 gives you math.MaxInt32 + 1, because math.MaxInt32 is not representable in float32.

The result of converting float32 to int32 on x86 is essentially whatever the cvttss2si instruction generates with a 32-bit register (in Go asm this instruction is REP CVTTSS2SIL). That instruction raises a "floating-point invalid exception", but since that exception is always masked in practice, it generates "the indefinite integer value" 0x80000000. I don't know off-hand what other processors do.

Note that int32(float32(math.MaxInt32 + 1)) will also give you 0x80000000. What is the right answer there? Overflow is overflow. The program looks weird because it is obscuring where the overflow occurs. The overflow occurs when converting from int32 to float32. Once you've done that conversion, the key information has been lost, and the value has overflowed. Converting back to int32 is just reflecting that overflow.

I don't really see what there is to do here.

[cznic]

@ianlancetaylor Sure, it's been stated before that the overflow occurs for all numbers >= math.MaxInt32 - 63.

I don't really see what there is to do here.

What could be possibly done, is to have such concrete limits available in math.

[ianlancetaylor]

Sure, it's been stated before that the overflow occurs for all numbers >= math.MaxInt32 - 63.

That's not precisely what I am saying. I am saying there is loss of precision in converting an int32 value i to float32 when i >= math.MaxInt32. That loss of precision gives you a float32 value that is == math.MaxInt32 + 1. There is no overflow yet. The overflow occurs when you convert that float32 value back to int32, but at that point the overflow is obvious. The real problem is not an overflow problem, it's a loss of precision problem.

So if you believe that, what you are suggesting is that we explicitly state in the math package the value above which converting an int32 to a float32 will cause overflow if that float32 is converted back to an int32. That seems like an odd concept to me. What would we name it? And the right value to use for an overflow check is not hard to write once you're aware of the problem: it's math.Nextafter32(float32(math.MaxInt32), 0)).

[griesemer]

As @cherrymui has stated above, the spec does say:

In all non-constant conversions involving floating-point or complex values, if the result type cannot represent the value the conversion succeeds but the result value is implementation-dependent.

As @ianlancetaylor explained, this is really the crux of the issue.

Perhaps the spec could be more precise; specifically I think that the notion of "representing a value" is not well defined. There's already an issue: #15389 .

I'm going to close this in favor of the above. It can always be re-opened if need be.

[cznic]

@ianlancetaylor The int32 to float32 conversion was definitely a poor choice of mine to use in the example, because I don't think I'm talking about that (direction) of the conversion at all. However, your example with math.Nextafter finally enligtened me, thank you. (Fun fact: Before seeing your example it would probably never occur to me that the after part of that function name can actually mean before - when the arguments are such that x > y. )

Edit: Added @ianlancetaylor as @griesemer closed the issue seconds before I wanted to do the same.