{{ message }}

Closed
Closed

8262744: Formatter '%g' conversion uses wrong format for BigDecimal rounding up to limits#3363

Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
Filter file types

Just for now

@@ -3821,12 +3821,10 @@ private void print(StringBuilder sb, BigDecimal value, Locale l,
else if (precision == 0)
prec = 1;

BigDecimal tenToTheNegFour = BigDecimal.valueOf(1, 4);
BigDecimal tenToThePrec = BigDecimal.valueOf(1, -prec);
value = value.round(new MathContext(prec));
This conversation was marked as resolved by igraves
if ((value.equals(BigDecimal.ZERO))
|| ((value.compareTo(tenToTheNegFour) != -1)
&& (value.compareTo(tenToThePrec) == -1))) {
|| ((value.compareTo(BigDecimal.valueOf(1, 4)) != -1)
&& (value.compareTo(BigDecimal.valueOf(1, -prec)) == -1))) {

stuart-marks Apr 21, 2021 Member

Note that `compareTo` in general specifies a negative, zero, or positive return value, but BigDecimal and BigInteger specify a return value of -1, 0, and 1. So the code here that compares against -1 is strictly correct. However, the BigDecimal/BigInteger.compareTo docs say "The suggested idiom..." is a relative comparison against zero.

Indeed, the BigDecimal::compareTo method does always seem to return -1, 0, or 1 so this code is not incorrect. Well, maybe. I checked quickly and the BigDecimal comparison logic is fairly intricate (and also runs through BigInteger) so I might have missed something. Also, BigDecimal is subclassable, so an override of `compareTo` might return something other than -1, 0, or 1, even though strictly speaking this would violate the BigDecimal spec.

I'm wondering if there should be a followup bug that changes these tests to `>= 0` and `< 0`.

int e = - value.scale()
+ (value.unscaledValue().toString().length() - 1);
ProTip! Use n and p to navigate between commits in a pull request.