Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
proposal: math/big: Decimal #12127
In many areas, especially in commerce, exact values need to be processed and the inputs are commonly decimal. Unfortunately, decimal values cannot be represented accurately using binary floating points.
Since Go can be used in many places where accurate decimal arithmetic is required it seems reasonable to support it as part of the standard library.
The idea is to add math/big Decimal data type that supports integer, fixed-point, and floating-point decimal numbers.
The Decimal API is consistent with Float API when possible.
A Decimal consists of a boolean sign, an arbitrary precision integer unscaled value and a 32-bit integer scale. The value of the number represented by the Decimal is therefore
A Decimal (similar to Float) may also be zero (+0, -0) or infinite (+Inf, -Inf). NaN values are not supported.
Each Decimal value also has a precision, rounding mode, and accuracy. The precision is the maximum number of decimal digits available to represent the value. The rounding mode specifies how a result should be rounded to fit into the unscaled value, and accuracy describes the rounding error with respect to the exact result.
The same numerical value can have different representations (with different scales). The rules of arithmetic and rounding specify both the numerical result and the scale used in the result's representation.
The Decimal type provides operations for
Implementation of the above operations closely follows the General Decimal Arithmetic Specification. This specification defines a decimal arithmetic which meets the requirements of commercial, financial, and human-oriented applications. It also matches the decimal arithmetic in the IEEE 754 Standard for Floating Point Arithmetic.
Test cases can be based on General Decimal Arithmetic
IEEE 754 fixed-point types
By setting the desired precision to 7, 16 or 34 and using matching rounding mode (typically ToNearestEven), Decimal operations produce the same results as the corresponding decimal32, decimal64 or decimal128 IEEE-754 arithmetic for operands that correspond to normal (i.e., not denormal) decimal32, decimal64 or decimal128 numbers.
I also miss a decimal floating point data type.
The reasons why decimal floating numbers are important can be found here:
In my projects, I use the C library decNumber, written by Mike Cowlishaw.
To download the source files, go to "International Components for Unicode (ICU)" and click "decNumber package".
The decQuad type is 128 bits long, and stores a decimal floating point number.
Personnally, I just put the .c and .h files in my project tree, and call the library functions using cgo.
Example of use:
It would be great if a Go package wrapping this C library could be integrated in Go standard library, before being replaced by a pure Go solution with the same API.
Note that this library is already integrated in gcc compiler.
math/big is all about arbitrary precision arithmetic, and in the case of Float, the precision can be set arbitrarily (within memory limits, essentially).
I'd be ok with a big.Decimal type; as long as it's also arbitrary precision. Fixed-size data types such as decQuad, decDouble belong into a separate package in my mind.
In fact, the Float implementation already uses some (minimal) decimal arithmetic for Float->string conversion; having an actual Decimal type would make that code unnecessary (in favor of using a fully-fledged Decimal implementation).
That said, I would like to see more "meat" around this proposal. With the Float implementation, the fixed-width IEEE floating-point spec provided reasonable guidance. What are the parameters for a Decimal implementation? Is it at Fixed-point implementation (arbitrary size number with fixed decimal point), or decimal floating-point representation (like Float, with mantissa and exponent, but the mantissa is always decimal)? Are the other options? What are the pros and cons? What are other packages doing? What is required for say business/banking applications?
Just a note: any fixed-size Decimal implementation should consider the IEEE 754-2008 specs for decimal floating point, and the corresponding C extensions for decimal floating point (http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1312.pdf).
@kostya-sh: do you really need arbitrary precision for financial computation ?
I say that because I think the API of big.Float (and big,Decimal will certainly be the same) is quite complex.
As an example, for addition:
You must prepare the result variable with proper precision and rounding mode, and think about it for each arithmetic operation.
With a fixed-size floating point API, and decimal128 data type, all you do would be:
where context contains the rounding mode, and stores computation error of it occurs.
The API of big.Float introduces a complexity that is not needed for most financial computation.
Usual range for monetary values
Assets under management for a large multinational insurance company is:
This figure is the sum of amounts that can have 8 digits after decimal point, because this precision is needed when working with foreign currency conversion.
This means that for a large company, a decimal data type must cope with figures like:
100,000,000,000.00000000, that is, 20 significant digits.
A decimal128 fixed-size data type provides 34 significant digits, which can easily store these figures.
When doing financial computation, intermediate calculation can require some more digits, but 34 significant digits is really enough for most real cases financial computation.
That's why I think that big.Decimal for most financial applications is overkill, and a fixed-size decimal128 base-10 floating type is more than "good enough".
PRO and CONS
CON: if like big.Float, the API is more complex than needed
PRO: API is simple
Decimal floating point package used by other languages:
Let a and b be operands, and z is the result.
C: gcc contains "decNumber" implementation of Mike Cowlishaw, fixed-size floating point (it is an extension of GNU C)
java.math.BigDecimal is arbitrary-precision, consisting of an arbitrary precision integer (like Go big.Int) and a scale.
Python "decimal" module is also arbitrary-precision, with an API that ressembles very much to the C "decNumber" API, because the IBM decimal implementation by Mike Cowlishaw was chosen
Microsoft C# "decimal"
That being said, any means of working with base-10 floating point numbers, be it arbitrary-precision or fixed-size, will be welcomed by all those that work with monetary values ;-)
@rin01, I agree that for the most (if not all) financial computations decimal128 is enough. This is approach is used by .NET.
However I think that arbitrary precision
The following proposal documents can be used to understand what other packages are doing:
Both Java and Python implementations are based on a decimal arithmetic model, as defined by the relevant standards: IEEE 854 , ANSI X3-274 and the proposed revision of IEEE 754.
While Java and Python implementations are slightly different they have the following commonalities:
In Go, numeric data types have both a built-in fixed-size implementation, and a package for arbitrary-precision.
Ultimately, Go should perhaps have both kinds.
If anyone wants to play with a fixed-size decimal package, I have written this Go wrapper around the decNumber package of Mike Cowllishaw, but just the "decQuad" data type.
decnum.Quad is a 128 bits fixed-size base-10 floating point
@rin01, your decnum package looks very interesting. In fact I think that a package that supports both fixed size types (
If possible I would like to keep this proposal limited to the new
I will update the proposal with more details as requested by @griesemer soon.
@kostya-sh: I agree with you. This proposal should be limited to math/big Decimal type with similar API to Float type. This would be great and will make the arbitrary-precision package math/big complete in a consistent manner.
Fixed-size base-10 floating point like decimal128 is another topic and should belong to another package and issue.
As an FYI, some of the issues that are aimed to be solved by this proposal, like rounding modes and precision (trailing zeros), I plan to address in the number formatting package in the text repo. In this model, aspects like rounding and precision are considered a property of the visualization, not the underlying number. This allows precision in computation to be higher than that of visualization. The number rendering would also support the big variants.
This is not to say that an addition of big.Dec (naming to be consistent) wouldn't be useful; I think it would be, even though it could be easily simulated with big.Int. Mostly, it would define a standard for decimal numbers so that it can be supported out of the box in other repos (like text). Note that many major frameworks, including .Net, OS X Foundation, Java etc. all have this functionality in their core library, which indicates it probably belongs there.
One could try to make the rounding definitions more general than just math/big. One could define a math/round package (as in .Net) or, IMO better, define the rounding modes and behaviors in math and add a func Round(float64, Rounding) (res float64, exact bool) to complement Floor and Trunc. In math/big, the various operations would use these rounding types. The method Dec(round math.Rounding) (d *Dec, exact bool) could then be defined on Rat to convert it to Dec. There is no efficient easy way to do that now that I can think of.
Note that these rounding parameters don't cover the whole spectrum of rounding algorithms. For currencies and in finance one sometimes does not round to a power of 10. I think it will be sufficient to simulate it, though. For example, rounding for Swiss accounting could be achieved by rounding HalfEven and then completing the rounding "manually" to select 0 or 5 for the last digit.
@mpvl, thank you for your feedback. Please find my comments below.
Precision and rounding mode should be available to operations on big.Decimal because of the following reasons:
Number of trailing zeroes is determined by the scale, not precision. The precision is the upper limit on the number of significant digits in a result. E.g. for 1.2000, scale is 4 and precision could be any number >=2.
I prefer Decimal name rather than Dec but I will leave the decision to golang team.
I agree that
Methods to convert between big.Decimal and float32, float64, big.Int, big.Float and big.Rat will be part of this proposal.
Various accounting rounding rules are not described in IEEE 754 standard and don't have to be supported by big.Decimal. It should be easy to implement them if necessary though. For example last 2 digits required for Swiss rounding (as described at (http://blog.syspro.com/2013/09/12/rounding-accounting) could be calculated by setting scale to 4 and getting reminder of division by 0.01.
@kostya-sh Regarding your question: Both the math and math/big package are subject to the Go compatibility rules - thus we can add stuff but in general we cannot change or remove existing API. One could define Accuracy and RoundingMode in math and redefine the same constants in math/big. But let's not get bogged down by implementation details for now.
I'm worried that this proposal's scope is growing rather than becoming more focused. There's two questions to be answered:
If the goal is to make Go a viable language for business/commercial applications (which I think would be a great goal), than a first step might be an easy-to-use, standards-compliant implementation of a fixed-point decimal package (from what I understand so far, 128bit seems plenty). Such a package doesn't need to be under math or math/big. It could be the base of a larger library of business-related packages.
If the goal is an arbitrary precision decimal type, I think it would simply be an extension of package big (as in big.Decimal).
I think the proposal should focus on one of the two, I'm not convinced that they need to be combined. My vote would be for making this about 1): A proposal for commercially viable decimal number/fixed-point package. This is the more interesting one, with potential for large impact if done well.
Instead of focusing on a concrete implementation, the proposal should define exactly what would be needed for commercial applications. Business/accounting software experts, anyone?
@griesemer as far as I know Accuracy and RoundingMode are new types in Go 1.5. The only reason I asked the question is that now (before 1.5 final release) is the only time they still can be moved without breaking compatibility. I don't have a strong opinion on the location of these types though and if you think that they are defined in the right package let them be.
I agree with you that the scope of this proposal should be limited. I saw a few questions on StackOverflow (1, 2, 3, 4, 5, 6) that would be very easy to answer if Go had big.Decimal type. That was my original motivation for creating this proposal.
I am by no means an expert but I have some experience working with financial (though not accounting) applications in Java. It provides arbitrary precision decimal class (java.math.BigDecimal) in the core library and it is recommended to use this class for monetary calculations. It also integrates nicely with decimal database types via JDBC API. These are the reasons I believe that
I would prefer to keep this proposal limited to big.Decimal and create another proposal for "a commercially viable decimal number/fixed-point package". It probably should be discussed on the mailing list as proposals on github issue tracker are visible only to the limited number of interested developers.
@kostya-sh The time for 1.5 API changes has long passed, especially with the final release any day now.
It's ok to make this about math/big.Decimal. In that case, I think the questions about Accuracy, RoundingMode are already answered. They are already where you'd want them.
So the rest of the proposal could then talk about internal representation, supported operations, etc.
@mpvl I'd be ok with a Decimal type in math/big, but I agree with @adg that prototyping it outside makes a lot of sense. I think the basic proposal (functionality, etc.) seems reasonable, but the devil is in the details, and some things may only become clear when an implementation is attempted.
An external implementation may be (slightly) less efficient since it cannot use say big.nat - but it's probably marginal. Prototyping it externally also removes any time pressure from getting it done for a specific release.
referenced this issue
Aug 26, 2015
referenced this issue
Apr 12, 2016
@ericlagergren if you want to work in golang.org/x/exp, then you need to find someone to review your code (and ideally that is someone who works on other parts of Go, already). But there's no reason to let that hold you back; you can just work on the code in a repository hosted anywhere.
I have not used any decimal implementations myself. In addition to @ericlagergren's, the next-to-last comment currently on #12332 reports success using https://github.com/shopspring/decimal. As usual, any code you rely upon in important, production systems should first undergo thorough testing to ensure that it meets your needs.
referenced this issue
Mar 30, 2017
I am still running into the need for larger precision exact decimal type. Google, while searching use cases, please realize Go is being used for many cryptocurrency projects. Half these projects appear to internally use float64s, half use shopspring.Decimal. Please support something similar to C++'s
@brobits Here are some alternatives I feel comfortable recommending (in a slightly biased order :-)
Most of the other decimal libraries don't have robust tests or skim on some fundamental aspects of the GDA spec. I share your sympathies.