math/big: GCD support for negative and zero arguments

[TuomLarsen]

Please consider adding support for zero and negative arguments to math/big GCD.

As of version 1.11.1, to calculate GCD of two integers one has to write:

new(big.Int).GCD(nil, nil, new(big.Int).Abs(a), new(big.Int).Abs(b))

In other words, one needs two extra allocations (or copies) as the nat field of big.Int is private.

If on the other hand negative inputs were allowed the code would read:

new(big.Int).GCD(nil, nil, a, b)

For negative inputs the resulting GCD could be treated as GCD(|a|, |b|), the precise sign does not really matter as long as it is documented. (For example, "GCD is non-negative".)

While I'm at it, it would be also nice to support zero arguments as it is well defined (GCD(a, 0) = |a|, GCD(0, 0) = 0) and it would simplify the code because there would be no need to special-case zero inputs. Currently it sets the result to zero.

[ianlancetaylor]

CC @griesemer

[bmkessler]

Note, you can avoid allocation by using:

new(big.Int).GCD(nil, nil, a.Abs(a), b.Abs(b))

Which switches the sign without allocation.

What would be the proposed convention for the signs of the Bezout coefficients?

z = a*x + b*y
or
z = |a|*x + |b|*y

I don't think there is an established convention here.

For zero values, I presume you would set x=0 if a=0 and y=0 if b=0

What is the use case for GCD of negative numbers?

[TuomLarsen]

That would mutate the arguments, however. I don't think it is desirable to change a or b when asking for new(big.Int).

If it worked as GCD(|a|, |b|) then I think the latter would make more sense. And as you say for the case of zeros.

Normalizing fractions, computing content of polynomials, simplify integer matrices, all these work beyond natural numbers.

[bmkessler]

Yes, it's not ideal that you would have to track the signs of a and b and restore them after calculating the GCD, I was just noting that you could currently calculate the GCD without allocating by doing some additional bookkeeping.

For reference, GMP uses the first convention for the Bezout coefficients z = a*x + b*y
https://gmplib.org/manual/Number-Theoretic-Functions.html

[TuomLarsen]

This as well but also accessing a or b from other threads would be unsafe.

[Merovius]

Disclaimer: No, don't do this ;)

func evilAbs(x *big.Int) *big.Int {
	y := *x
	y.Abs(&y)
	return &y
}

[gopherbot]

Change https://golang.org/cl/164972 mentions this issue: math/big: allow all values for GCD

[bcmills]

The documentation for (*big.Int).GCD currently says:

If either a or b is <= 0, GCD sets z = x = y = 0.

That's unfortunate: it gives well-defined non-error behavior for an input that arguably should have been defined to be a user error. It also makes any change to GCD for zero or negative inputs violate Go 1 compatibility, and per #28221, we want to avoid such redefinitions.

That seems to suggest that the extended GCD function must have a new name.

[bmkessler]

That is unfortunate. I don't think there should be a different GCD function that accepts zero or negative inputs. So that probably makes this a Go 2 proposal then.

[bcmills]

Even a Go 2 proposal will generally need to follow the guidelines from #28221 in order to be accepted.
I don't think there is a viable alternative to putting the behavior in a separate method.

[bmkessler]

Doesn't math/big fall under standard-library-changes and not changes to the core language.

One of the benefits of a Go 2 transition is the chance to release some of the standard library packages from the Go 1 compatibility guarantee.

In particular, it would appear to be part of the penumbra-standard-library

The penumbra standard library consists of those packages that are included with a release but are maintained independently. This will be most of the current standard library. These packages will follow the same discipline as today, with the option to move to a v2 where appropriate.

Which seems to indicate this change would be possible in a future release.