Skip to content

Commit

Permalink
Change how math rounds integer results
Browse files Browse the repository at this point in the history
We need our `math` builtin to behave like `bc` with respect to rounding
floating point values to integer to avoid breaking to many existing
uses. So when scale is zero round down to the nearest integer.

Another change for #3157.
  • Loading branch information
Kurtis Rader committed Aug 24, 2017
1 parent 24d251f commit b816cd6
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 2 deletions.
2 changes: 1 addition & 1 deletion doc_src/math.txt
Expand Up @@ -15,7 +15,7 @@ The `math` command can evaluate multiple expressions separated by commas. The re

The following options are available:

- `-sN` or `--scale=N` sets the scale of the result. `N` must be an integer and defaults to zero (rounded to the nearest integer).
- `-sN` or `--scale=N` sets the scale of the result. `N` must be an integer and defaults to zero. A scale of zero causes results to be rounded down to the nearest integer. So `3/2` returns `1` rather than `2` which `1.5` would normally round to. This is for compatibility with `bc` which was the basis for this command prior to fish 3.0.0. Scale values greater than zero causes the result to be rounded using the usual rules to the specified number of decimal places.

\subsection return-values Return Values

Expand Down
6 changes: 5 additions & 1 deletion src/builtin_math.cpp
Expand Up @@ -133,7 +133,11 @@ static int evaluate_expression(wchar_t *cmd, parser_t &parser, io_streams_t &str
int nNum;
mu::value_type *v = p.Eval(nNum);
for (int i = 0; i < nNum; ++i) {
streams.out.append_format(L"%.*lf\n", opts.scale, v[i]);
if (opts.scale == 0) {
streams.out.append_format(L"%ld\n", static_cast<long>(v[i]));
} else {
streams.out.append_format(L"%.*lf\n", opts.scale, v[i]);
}
}

return STATUS_CMD_OK;
Expand Down

0 comments on commit b816cd6

Please sign in to comment.