New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Sign on result of GCD is unpredictable #58
Comments
I would expect the result of From a quick look at the Zarith code, the only case where a negative result can occur is when one of the arguments is 0, in which case the other argument is returned, while maybe its absolute value should be returned. (To be discussed.) |
Yes, I think we should follow mpz_gcd and always return a non-negative result. So, gcd(a,0) = gcd(0,a) = abs(a). Thus, the case for large values of a needs fixing. Also, I now realize that we are only fixing gcd. Maybe gcdext and lcm also need fixing for negative or nul arguments. |
See PR #59 for a fix for gcd, and also lcm, gcdext. |
- fix for issue #58: gcd, lcm, gcdext now behave as gmp for negative arguments - improved abs for nonnegative arguments
Merged! |
In particular, behavior of `Z.gcd` and `Z.lcm` has been fixed in 1.10.0, see ocaml/Zarith#58
In particular, behavior of `Z.gcd` and `Z.lcm` has been fixed in 1.10.0, see ocaml/Zarith#58
In particular, behavior of `Z.gcd` and `Z.lcm` has been fixed in 1.10, see ocaml/Zarith#58
In particular, behavior of `Z.gcd` and `Z.lcm` has been fixed in 1.10, see ocaml/Zarith#58
When numbers are small, the result of a call to gcd is positive when one parameter is zero, and the other parameter is negative.
However, when the negative value is "large", it returns a negative number.
At first glance, it appears that the "normalization" in the small-case happens here:
https://github.com/ocaml/Zarith/blob/master/caml_z.c#L1743
The text was updated successfully, but these errors were encountered: