Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rounding and other sources of computational approximations #27

Closed
dumblob opened this issue Feb 12, 2021 · 4 comments
Closed

Rounding and other sources of computational approximations #27

dumblob opened this issue Feb 12, 2021 · 4 comments

Comments

@dumblob
Copy link

dumblob commented Feb 12, 2021

Could you document how rounding works with DEC64? I've only found this:

; Rounding is to the nearest value. Ties are rounded away from zero. Integer
; division is floored. The result of modulo has the sign of the divisor.
; There is no negative zero.

And how about other sources of approximations (at least all those mentioned in vlang/v#5180 (comment) )? How they're handled in DEC64?

@douglascrockford
Copy link
Owner

That is the rule. What else do you need?

@dumblob
Copy link
Author

dumblob commented Feb 12, 2021

I wanted to see how comparisons are done (whether there is any influence of the "surrounding" like in IEEE 754, how are denormals/subnormals handled, etc.). But I found them right now farther in the asm source. So I think the first question is answered.

Could you maybe comment on other "sources of approximation" other than using the mantissa & exponent way of calculation (i.e. non-constant distribution of numbers on the scale of representable numbers)?

@douglascrockford
Copy link
Owner

Any floating point system will fail to represent exactly some rational numbers. DEC64 can exactly represent all of decimal fractions up to 16 or 17 significant digits. Binary floating point can not. Those are the numbers that are most important to humanity, because that is how humans understand numbers.

DEC64 does not have subnormals. No sticky bits. No modes. There is a lot of weirdness and complexity in IEEE 754 which is lacking in DEC64.

Comparisons are straightforward. Equality is an exact comparison.

I still do not understand what you are asking for.

@dumblob
Copy link
Author

dumblob commented Feb 13, 2021

Any floating point system will fail to represent exactly some rational numbers. DEC64 can exactly represent all of decimal fractions up to 16 or 17 significant digits. Binary floating point can not. Those are the numbers that are most important to humanity, because that is how humans understand numbers.

DEC64 does not have subnormals. No sticky bits. No modes. There is a lot of weirdness and complexity in IEEE 754 which is lacking in DEC64.

Comparisons are straightforward. Equality is an exact comparison.

I still do not understand what you are asking for.

Yep, if this is an exhaustive list of (non-)features of DEC64, then it answers my question 😉. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants