math/big: infinite loop in Int.ModSqrt for p = 1

[sylvainpelissier]

When passing p=1 in ModSqrt function, the code goes in infinite loop:

go/src/math/big/int.go

Line 940 in 3463852

for Jacobi(&n, p) != -1 {

The Jacobi symbol will gives one for ever.

For example: https://go.dev/play/p/37ExZ6Y-Hdp

[ALTree]

It does indeed go into an infinite loop. Thanks for reporting.

The method's documentation says

The modulus p must be an odd prime.

and 1 is not a prime, but going into an infinite loop is still not the best way to handle an invalid parameter.

[ALTree]

cc @griesemer @FiloSottile as per owners.

[gopherbot]

Change https://go.dev/cl/394077 mentions this issue: math/big: infinite loop in Int.ModSqrt for p = 1

[hopehook]

Other examples of causing infinite loop in Int.ModSqrt:

• p = -1
• p = -9
• p = 9

[gopherbot]

Change https://go.dev/cl/402457 mentions this issue: math/big: limit search for non-square in ModSqrt

[ianlancetaylor]

Rolling forward to 1.20.