cmd/compile: prove pass doesn't draw signed int conclusions from range checks

[josharian]

This might be fixed by outstanding prove CLs, but just in case, consider this snippet from math/expm1.go:

	var k int

	// ...

	switch {
	case k == -1:
		return 0.5*(x-e) - 0.5
	case k == 1:
		if x < -0.25 {
			return -2 * (e - (x + 0.5))
		}
		return 1 + 2*(x-e)
	case k <= -2 || k > 56: // suffice to return exp(x)-1
		y := 1 - (e - x)
		y = Float64frombits(Float64bits(y) + uint64(k)<<52) // add k to y's exponent
		return y - 1
	}
	if k < 20 {
		t := Float64frombits(0x3ff0000000000000 - (0x20000000000000 >> uint(k))) // t=1-2**-k
		y := t - (e - x)
		y = Float64frombits(Float64bits(y) + uint64(k)<<52) // add k to y's exponent
		return y
	}

Note the switch case k <= -2 || k > 56. The compiler rewrites this into a single unsigned comparison: uint(x) > 57. See func walkinrange in walk.go.

After the k < 20 check (in particular, when considering uint64(k)<<52, the prove pass has signed limits of [-9223372036854775808, 19] for k and no unsigned limits. But I believe that we should know that k is in [-2, 19].

I might have the culprit/analysis wrong...but in any case, we should know more about uint64(k) here.

Noticed while exploring #25087.

cc @rasky

[josharian]

@zdjones does your outstanding CL fix this issue as well?

[josharian]

Punting to 1.14 in any case.