Skip to content
This repository has been archived by the owner on Apr 23, 2021. It is now read-only.

Is this library correct in terms of Base10 calculations? #14

Closed
joeventura opened this issue Mar 22, 2018 · 1 comment
Closed

Is this library correct in terms of Base10 calculations? #14

joeventura opened this issue Mar 22, 2018 · 1 comment

Comments

@joeventura
Copy link

I am exploring to use this library in a financial application where base 10 calculations must be correct within 10 decimal paces.

System.Decimal provides correct base 10 math within its precision range, I believe 28 decimal places.

This library is very exciting because many applications can benefit from a faster, yet correct numeric data type.

How does this library compare to System.Decimal in terms of correctness?

@asik
Copy link
Owner

asik commented Mar 22, 2018

The precision of this type is 2^-32, which is approx 2,3283e-10. So it's not quite 10 decimals.
Basic arithmetic (+, -, *, /, %) is all correct within the precision of the type and well-tested.
Other operations have various degrees of accuracy which are usually noted in the description.
Note that the type performs saturation arithmetic which does not preserve mathematical properties once you run into its limits. (All numerical types eventually break down but saturation arithmetic can be tricky to deal with).

That said, I would not recommend using this as a faster replacement of System.Decimal (assuming it's faster...). This type is intended to replace System.Single or System.Double in applications where determinism is important or floating-point hardware is not available.

Note that since it's based on System.Int64, it will perform significantly faster in a 64-bit application where that type fits into registers.

@asik asik closed this as completed Apr 4, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants