-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Clarification on what "precision" means in roundingPriority: morePrecision/lessPrecision #96
Comments
Thank you for the issue! I agree that the result is a bit counter-intuitive. Here is my mental model. The choice between "more precision" and "less precision" is only made with respect to the maximum fraction or significant digits. The minimum digits, which only control trailing zeros, play no part in the "more precision" or "less precision" game. Since maximum significant digits default to 21 and maximum fraction digits normally default to only 3, significant digits will almost always "win" on morePrecision mode unless they are overridden with the In other words, the surprising behavior only happens when minimum digits are set in the absence of maximum digits being set. This algorithm was chosen because it solves our known use cases and reads cleanly in the spec. That said, I am open to exploring alternative algorithms, although doing so would need to meet a high bar since this is a Stage 3 proposal. |
I'm not sure anything would need to change, if only the meaning of "more precision" were explained more clearly in the document. Maybe " Alternatively if you did explore alternative algorithms, for reference here is roughly what I expected when I read the current documentation:
|
TC39-TG2 discussion: https://github.com/tc39/ecma402/blob/master/meetings/notes-2022-06-16.md#clarification-on-what-precision-means-in-roundingpriority-moreprecisionlessprecision-96 I plan to draft up the change both here and in ICU to make sure that it behaves the way we expect and satisfies all of the original use cases. |
OK, I drafted unicode-org/icu#2260 in ICU to see the impact of the proposed change. What I found: it appears that all ICU tests pass (have the current behavior) both before and after this change; no expectations needed to be updated. I added a new test where I was able to demonstrate the change in behavior. This tells me two things:
We already know from experience that there really isn't a strong use case behind setting minimum digits without setting maximum digits. If you set both, this is a non-issue. The Test262 test is just fairly esoteric. I am therefore inclined to update the docs without touching the spec or implementation. |
Hi, I landed here through reviewing the test262 tests for this proposal, specifically this change: tc39/test262#3506
In short it's about the following code:
There's a conflict here that
roundingPriority
is supposed to resolve. If we takeminimumSignificantDigits
the result is "1.0". If we takeminimumFractionDigits
the result is "1.00". The test262 test asserts that "1.0" is the correct result.@romulocintra and I have stepped through the algorithm in the spec, on paper, and have verified that that test262 test conforms. The correct result according to the spec is "1.0". However, if I read the proposal's documentation I'm not really understanding why "1.0" should be correct. Here's what the documentation says:
With my user hat on, who reads the documentation but not the spec, I'd expect that "1.00" has more precision (3 significant digits) than "1.0" (2 significant digits), so the outcome should be that "1.00" wins the conflict.
Where is my understanding faulty? My guess is that the documentation is using a different meaning of "precision" than what I expected. Could the meaning of "precision" be clarified in the documentation?
(Note the documentation actually has this example in it already, but it doesn't mention what the result is 😄)
The text was updated successfully, but these errors were encountered: