math/big: ModSqrt returns wrong value when aliasing parameters

[FiloSottile]

What version of Go are you using (go version)?

go 1.9

What operating system and processor architecture are you using (go env)?

• darwin_amd64
• Playground

What did you do?

Aliased z and x in ModSqrt, as allowed by the package docs and not disallowed by the function docs.

y.ModSqrt(y, p.P)

https://play.golang.org/p/MsOief8jlg

What did you expect to see?

The same result as new(big.Int).ModSqrt(y, p.P).

What did you see instead?

An incorrect value is computed.

[FiloSottile]

The optimized path modSqrt3Mod4Prime introduced by ea0491b starts with z.Set(p), which when aliasing will improperly set x = p, too.

[gopherbot]

Change https://golang.org/cl/70832 mentions this issue: math/big: fix ModSqrt optimized path for x = z