Skip to content

Commit

Permalink
Round decimal division of tuple elements.
Browse files Browse the repository at this point in the history
When performing tuple division, the floating point representation of
decimal divisors may yield surprising results.  For example, the
following expression

  >> 1.1.1 / 0.1

can return 9.9.9 or 10.10.10 depending on the host platform.

To address such discrepancies, `round(3)` before casting the result of
tuple element division to a `REBINT`; this should improve cross-platform
consistency.
  • Loading branch information
0branch committed Mar 13, 2013
1 parent fc51038 commit 7b4e852
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/core/t-tuple.c
Original file line number Diff line number Diff line change
Expand Up @@ -232,7 +232,7 @@
case A_DIVIDE:
if (IS_DECIMAL(arg) || IS_PERCENT(arg)) {
if (dec == 0.0) Trap0(RE_ZERO_DIVIDE);
v=(REBINT)(v/dec);
v=(REBINT)round(v/dec);
} else {
if (a == 0) Trap0(RE_ZERO_DIVIDE);
v /= a;
Expand Down

0 comments on commit 7b4e852

Please sign in to comment.