-
Notifications
You must be signed in to change notification settings - Fork 17.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
proposal: math/big: Decimal #12127
Comments
I also miss a decimal floating point data type. The reasons why decimal floating numbers are important can be found here: In my projects, I use the C library decNumber, written by Mike Cowlishaw.
To download the source files, go to "International Components for Unicode (ICU)" and click "decNumber package". The decQuad type is 128 bits long, and stores a decimal floating point number. Personnally, I just put the .c and .h files in my project tree, and call the library functions using cgo. Example of use: It would be great if a Go package wrapping this C library could be integrated in Go standard library, before being replaced by a pure Go solution with the same API. Note that this library is already integrated in gcc compiler.
|
math/big is all about arbitrary precision arithmetic, and in the case of Float, the precision can be set arbitrarily (within memory limits, essentially). I'd be ok with a big.Decimal type; as long as it's also arbitrary precision. Fixed-size data types such as decQuad, decDouble belong into a separate package in my mind. In fact, the Float implementation already uses some (minimal) decimal arithmetic for Float->string conversion; having an actual Decimal type would make that code unnecessary (in favor of using a fully-fledged Decimal implementation). That said, I would like to see more "meat" around this proposal. With the Float implementation, the fixed-width IEEE floating-point spec provided reasonable guidance. What are the parameters for a Decimal implementation? Is it at Fixed-point implementation (arbitrary size number with fixed decimal point), or decimal floating-point representation (like Float, with mantissa and exponent, but the mantissa is always decimal)? Are the other options? What are the pros and cons? What are other packages doing? What is required for say business/banking applications? |
Just a note: any fixed-size Decimal implementation should consider the IEEE 754-2008 specs for decimal floating point, and the corresponding C extensions for decimal floating point (http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1312.pdf). |
@kostya-sh: do you really need arbitrary precision for financial computation ? I say that because I think the API of big.Float (and big,Decimal will certainly be the same) is quite complex. As an example, for addition:
You must prepare the result variable with proper precision and rounding mode, and think about it for each arithmetic operation. With a fixed-size floating point API, and decimal128 data type, all you do would be:
where context contains the rounding mode, and stores computation error of it occurs. The API of big.Float introduces a complexity that is not needed for most financial computation. Usual range for monetary valuesAssets under management for a large multinational insurance company is: $265,507,000,000 This figure is the sum of amounts that can have 8 digits after decimal point, because this precision is needed when working with foreign currency conversion. This means that for a large company, a decimal data type must cope with figures like: 100,000,000,000.00000000, that is, 20 significant digits. A decimal128 fixed-size data type provides 34 significant digits, which can easily store these figures. When doing financial computation, intermediate calculation can require some more digits, but 34 significant digits is really enough for most real cases financial computation. That's why I think that big.Decimal for most financial applications is overkill, and a fixed-size decimal128 base-10 floating type is more than "good enough". PRO and CONSbig.Decimal arbitrary-precision CON: if like big.Float, the API is more complex than needed decnumber.Decimal128 fixed-size PRO: API is simple Decimal floating point package used by other languages:Let a and b be operands, and z is the result. C: gcc contains "decNumber" implementation of Mike Cowlishaw, fixed-size floating point (it is an extension of GNU C)
java.math.BigDecimal is arbitrary-precision, consisting of an arbitrary precision integer (like Go big.Int) and a scale.
Python "decimal" module is also arbitrary-precision, with an API that ressembles very much to the C "decNumber" API, because the IBM decimal implementation by Mike Cowlishaw was chosen
Microsoft C# "decimal"
That being said, any means of working with base-10 floating point numbers, be it arbitrary-precision or fixed-size, will be welcomed by all those that work with monetary values ;-) |
@rin01, I agree that for the most (if not all) financial computations decimal128 is enough. This is approach is used by .NET. However I think that arbitrary precision
|
The following proposal documents can be used to understand what other packages are doing: Reference documentation: Java BigDecimal, Python decimal Both Java and Python implementations are based on a decimal arithmetic model, as defined by the relevant standards: IEEE 854 , ANSI X3-274 and the proposed revision of IEEE 754. While Java and Python implementations are slightly different they have the following commonalities:
Python
|
In Go, numeric data types have both a built-in fixed-size implementation, and a package for arbitrary-precision. Ultimately, Go should perhaps have both kinds.
|
If anyone wants to play with a fixed-size decimal package, I have written this Go wrapper around the decNumber package of Mike Cowllishaw, but just the "decQuad" data type. decnum.Quad is a 128 bits fixed-size base-10 floating point https://github.com/rin01/decnum |
@rin01, your decnum package looks very interesting. In fact I think that a package that supports both fixed size types ( If possible I would like to keep this proposal limited to the new I will update the proposal with more details as requested by @griesemer soon. |
I have updated the issue to include more details about the proposed implementation. |
@kostya-sh: I agree with you. This proposal should be limited to math/big Decimal type with similar API to Float type. This would be great and will make the arbitrary-precision package math/big complete in a consistent manner. Fixed-size base-10 floating point like decimal128 is another topic and should belong to another package and issue. |
As an FYI, some of the issues that are aimed to be solved by this proposal, like rounding modes and precision (trailing zeros), I plan to address in the number formatting package in the text repo. In this model, aspects like rounding and precision are considered a property of the visualization, not the underlying number. This allows precision in computation to be higher than that of visualization. The number rendering would also support the big variants. This is not to say that an addition of big.Dec (naming to be consistent) wouldn't be useful; I think it would be, even though it could be easily simulated with big.Int. Mostly, it would define a standard for decimal numbers so that it can be supported out of the box in other repos (like text). Note that many major frameworks, including .Net, OS X Foundation, Java etc. all have this functionality in their core library, which indicates it probably belongs there. One could try to make the rounding definitions more general than just math/big. One could define a math/round package (as in .Net) or, IMO better, define the rounding modes and behaviors in math and add a func Round(float64, Rounding) (res float64, exact bool) to complement Floor and Trunc. In math/big, the various operations would use these rounding types. The method Dec(round math.Rounding) (d *Dec, exact bool) could then be defined on Rat to convert it to Dec. There is no efficient easy way to do that now that I can think of. Note that these rounding parameters don't cover the whole spectrum of rounding algorithms. For currencies and in finance one sometimes does not round to a power of 10. I think it will be sufficient to simulate it, though. For example, rounding for Swiss accounting could be achieved by rounding HalfEven and then completing the rounding "manually" to select 0 or 5 for the last digit. |
@mpvl, thank you for your feedback. Please find my comments below. Precision and rounding mode should be available to operations on big.Decimal because of the following reasons:
Number of trailing zeroes is determined by the scale, not precision. The precision is the upper limit on the number of significant digits in a result. E.g. for 1.2000, scale is 4 and precision could be any number >=2. I prefer Decimal name rather than Dec but I will leave the decision to golang team. I agree that Methods to convert between big.Decimal and float32, float64, big.Int, big.Float and big.Rat will be part of this proposal. Various accounting rounding rules are not described in IEEE 754 standard and don't have to be supported by big.Decimal. It should be easy to implement them if necessary though. For example last 2 digits required for Swiss rounding (as described at (http://blog.syspro.com/2013/09/12/rounding-accounting) could be calculated by setting scale to 4 and getting reminder of division by 0.01. |
@kostya-sh Regarding your question: Both the math and math/big package are subject to the Go compatibility rules - thus we can add stuff but in general we cannot change or remove existing API. One could define Accuracy and RoundingMode in math and redefine the same constants in math/big. But let's not get bogged down by implementation details for now. I'm worried that this proposal's scope is growing rather than becoming more focused. There's two questions to be answered:
If the goal is to make Go a viable language for business/commercial applications (which I think would be a great goal), than a first step might be an easy-to-use, standards-compliant implementation of a fixed-point decimal package (from what I understand so far, 128bit seems plenty). Such a package doesn't need to be under math or math/big. It could be the base of a larger library of business-related packages. If the goal is an arbitrary precision decimal type, I think it would simply be an extension of package big (as in big.Decimal). I think the proposal should focus on one of the two, I'm not convinced that they need to be combined. My vote would be for making this about 1): A proposal for commercially viable decimal number/fixed-point package. This is the more interesting one, with potential for large impact if done well. Instead of focusing on a concrete implementation, the proposal should define exactly what would be needed for commercial applications. Business/accounting software experts, anyone? |
@griesemer as far as I know Accuracy and RoundingMode are new types in Go 1.5. The only reason I asked the question is that now (before 1.5 final release) is the only time they still can be moved without breaking compatibility. I don't have a strong opinion on the location of these types though and if you think that they are defined in the right package let them be. I agree with you that the scope of this proposal should be limited. I saw a few questions on StackOverflow (1, 2, 3, 4, 5, 6) that would be very easy to answer if Go had big.Decimal type. That was my original motivation for creating this proposal. I am by no means an expert but I have some experience working with financial (though not accounting) applications in Java. It provides arbitrary precision decimal class (java.math.BigDecimal) in the core library and it is recommended to use this class for monetary calculations. It also integrates nicely with decimal database types via JDBC API. These are the reasons I believe that
I would prefer to keep this proposal limited to big.Decimal and create another proposal for "a commercially viable decimal number/fixed-point package". It probably should be discussed on the mailing list as proposals on github issue tracker are visible only to the limited number of interested developers. |
@kostya-sh The time for 1.5 API changes has long passed, especially with the final release any day now. It's ok to make this about math/big.Decimal. In that case, I think the questions about Accuracy, RoundingMode are already answered. They are already where you'd want them. So the rest of the proposal could then talk about internal representation, supported operations, etc. |
I have updated the original proposal with more details as requested. |
Why not implement this as an external package? Then we can see how it looks and how generally useful it is before talking about including it in the standard library. |
@adg Having a big.Decimal in core allows other core packages, like database/sql to refer to it. You won't have this benefit with an external package. In general standardizing on this type seems quite useful. |
@mpvl I'd be ok with a Decimal type in math/big, but I agree with @adg that prototyping it outside makes a lot of sense. I think the basic proposal (functionality, etc.) seems reasonable, but the devil is in the details, and some things may only become clear when an implementation is attempted. An external implementation may be (slightly) less efficient since it cannot use say big.nat - but it's probably marginal. Prototyping it externally also removes any time pressure from getting it done for a specific release. |
@griesemer , I agree with your points. I've created https://github.com/kostya-sh/go-decimal-proposal and will work on it my spare time. Once it is more or less ready I will update this issue |
@kostya-sh , have you looked at this package http://godoc.org/speter.net/go/exp/math/dec/inf ? https://groups.google.com/forum/#!topic/golang-nuts/Zu_gSp56N4Y |
If one was to begin working on an exp/decimal, where would one begin? |
@ericlagergren if you want to work in golang.org/x/exp, then you need to find someone to review your code (and ideally that is someone who works on other parts of Go, already). But there's no reason to let that hold you back; you can just work on the code in a repository hosted anywhere. |
@adg I have github.com/ericlagergren/decimal I've submitted code to the stdlib but I didn't know if creating something in /x/exp required anything other than a CL. |
@ericlagergren yep, it's just a matter of sending a CL. But there needs to be at least one other person willing to work with you on it (do code reviews, etc). |
@adg I threw up a CL if you're at all interested :-) |
CL https://golang.org/cl/25146 mentions this issue. |
@ericlagergren thanks for doing that, but I don't have the bandwidth to review it. |
@adg no worries, just in case you were curious is all. Cheers.
|
I took a look at the CL, and replied there. An initial CL should be more substantial: it should include the basic API and documentation, at least. |
@adg The API has been outlined with comments for each routine. Apologies for the lacking CL. I've a ton more code to throw up whenever it's needed. |
@ericlagergren thanks. That looks like a fine start. However, as I said earlier, you really need another interested contributor to work with you on this. Otherwise there's no difference to just using the one you already have on your GitHub account. |
@adg yep, I've been meaning to make a post on golang-nuts (or golang-dev?) when I have a free minute. Thanks for giving the CL a look over, I appreciate it. |
For now we would like to see development continue outside the main repository, with the understanding that if more code starts to use decimal floats we can reconsider adding them somewhere more standard. |
Is there any de facto library outside the main repository you can recommend @rsc? |
@ruimarinho there's https://github.com/ericlagergren/decimal which I wrote, although I'm not trying to give out russ' blessings for him 😄 |
I have not used any decimal implementations myself. In addition to @ericlagergren's, the next-to-last comment currently on #12332 reports success using https://github.com/shopspring/decimal. As usual, any code you rely upon in important, production systems should first undergo thorough testing to ensure that it meets your needs. |
which one did you use @ruimarinho ? |
I am still running into the need for larger precision exact decimal type. Google, while searching use cases, please realize Go is being used for many cryptocurrency projects. Half these projects appear to internally use float64s, half use shopspring.Decimal. Please support something similar to C++'s |
@brobits Here are some alternatives I feel comfortable recommending (in a slightly biased order :-)
Most of the other decimal libraries don't have robust tests or skim on some fundamental aspects of the GDA spec. I share your sympathies. |
Motivation
In many areas, especially in commerce, exact values need to be processed and the inputs are commonly decimal. Unfortunately, decimal values cannot be represented accurately using binary floating points.
Since Go can be used in many places where accurate decimal arithmetic is required it seems reasonable to support it as part of the standard library.
Proposal
The idea is to add math/big Decimal data type that supports integer, fixed-point, and floating-point decimal numbers.
The Decimal API is consistent with Float API when possible.
Internal representation
A Decimal consists of a boolean sign, an arbitrary precision integer unscaled value and a 32-bit integer scale. The value of the number represented by the Decimal is therefore
sign × unscaledValue × 10^(-scale)
.A Decimal (similar to Float) may also be zero (+0, -0) or infinite (+Inf, -Inf). NaN values are not supported.
Each Decimal value also has a precision, rounding mode, and accuracy. The precision is the maximum number of decimal digits available to represent the value. The rounding mode specifies how a result should be rounded to fit into the unscaled value, and accuracy describes the rounding error with respect to the exact result.
The same numerical value can have different representations (with different scales). The rules of arithmetic and rounding specify both the numerical result and the scale used in the result's representation.
Operations
The Decimal type provides operations for
Implementation of the above operations closely follows the General Decimal Arithmetic Specification. This specification defines a decimal arithmetic which meets the requirements of commercial, financial, and human-oriented applications. It also matches the decimal arithmetic in the IEEE 754 Standard for Floating Point Arithmetic.
Test cases can be based on General Decimal Arithmetic
Testcases.
IEEE 754 fixed-point types
By setting the desired precision to 7, 16 or 34 and using matching rounding mode (typically ToNearestEven), Decimal operations produce the same results as the corresponding decimal32, decimal64 or decimal128 IEEE-754 arithmetic for operands that correspond to normal (i.e., not denormal) decimal32, decimal64 or decimal128 numbers.
Q&A
See https://www.python.org/dev/peps/pep-0327/#why-floating-point
See http://speleotrove.com/decimal/decifaq1.html#tzeros and http://speleotrove.com/decimal/decifaq4.html#unnari
See http://speleotrove.com/decimal/decifaq1.html#needed
In particular there is one example where decimal128 doesn't provide enough precision
See p3.
Other use-cases: big.Float to string conversion, representing arbitrary precision numbers that are supported in other systems such as databases (for example PostgreSQL decimal can store up to 131072 digits before the decimal point and up to 16383 digits after the decimal point)
Existing packages
The text was updated successfully, but these errors were encountered: