Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: spec: add decimal float types (IEEE 754-2008) #19787

Open
typeless opened this issue Mar 30, 2017 · 25 comments
Open

proposal: spec: add decimal float types (IEEE 754-2008) #19787

typeless opened this issue Mar 30, 2017 · 25 comments

Comments

@typeless
Copy link

@typeless typeless commented Mar 30, 2017

I know there was a proposal about decimal as an std package.
But according to https://en.wikipedia.org/wiki/Decimal_floating_point, there have already been multiple compliant hardware implementations, and supported by several commonly used programming languages. I think having decimal as first-class citizen like the other part of the IEEE-754 standard makes sense.

The upside is we don't have to spend the effort on API design upfront to make it useful like math.big. We can just limit the scope of the operations only within the Go's existing arithmetic operators. Designing a good numerical/arithmetic API beyond the conventional operators is harder and can be deferred later with no harm.

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Mar 30, 2017

Related: #12127, #19770

@gopherbot gopherbot added this to the Proposal milestone Mar 30, 2017
@gopherbot gopherbot added the Proposal label Mar 30, 2017
@bronze1man
Copy link
Contributor

@bronze1man bronze1man commented Mar 31, 2017

I think this proposal can be implement outside golang runtime, builtin and builtin package.
Does any implement exist?

@typeless
Copy link
Author

@typeless typeless commented Mar 31, 2017

@bronze1man Yes, there are thirty-party packages and proposals for including a decimal package into std. But this proposal is about adding fixed-size decimal floating point types, along with the existing binary floating point types using the existing arithmetic operators, rather than introducing a new package.

@rsc rsc added the Go2 label Apr 3, 2017
@rsc rsc changed the title proposal: builtin decimal types (IEEE 754-2008) proposal: spec: add decimal float types (IEEE 754-2008) Jun 16, 2017
@deanveloper
Copy link

@deanveloper deanveloper commented Jul 30, 2018

I made my own post for this without seeing this one first. Here are the reasons I believe that a decimal float type would be good for Go - #26699

@deanveloper
Copy link

@deanveloper deanveloper commented Sep 10, 2018

Adding onto this - If we had operator overloading, this could be implemented as a 3rd party lib instead

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Feb 12, 2019

Some useful information in:
http://open-std.org/JTC1/SC22/WG21/docs/papers/2014/n3871.html

In particular, this reference was useful:

The need for support of exact decimal computations is recognized in many communities and supported in several systems, although different alternatives for the support are chosen. Below is a list of example programming languages with decimal support:

  • The C committee is working on a Decimal TR as TR 24732. The decimal support in C uses built-in types _Decimal32, _Decimal64, and _Decimal128.
  • Java provides decimal arithmetic by java.math.BigDecimal, an arbitrary sized integer with an integer scale for the decimal places.
  • Python provides decimal.Decimal which is a fixed point decimal representation. The number of decimal digits can be set globally.
  • .Net provides System.Decimal which is a 128 bit decimal floating point. The details of this represntation are slightly different from the 128 bit decimal floating point in IEEE 754–2008. System.Decimal is accessible in C# as decimal.
  • SQL provides a fixed point decimal representation where the number of digits and the number of fractional digits can be chosen for each context.
  • Ruby provides BigDecimal, an arbitrary sized integer with an integer scale for the decimal places.
@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Feb 12, 2019

It would help to have a clear idea of who would want to use these new types.

It would also help if the proposal spelled out the new types and how untyped constants would be handled.

@ericlagergren
Copy link
Contributor

@ericlagergren ericlagergren commented Feb 12, 2019

It would help to have a clear idea of who would want to use these new types.

From manually polling people who use my (big) decimal package:

  • Banks
  • CAD
  • Numerical analysis
  • Databases (see: cockroachdb/apd)
  • E-commerce sites
  • Bitcoin sites
@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Feb 12, 2019

Thanks! But we shouldn't confuse big decimals with this proposal, which is for fixed size decimals. It's not obvious to me that people who use the former will want to use the latter. For example, the fact that people use the math/big Rat type does not suggest that we should add a rational number type to the language.

@ericlagergren
Copy link
Contributor

@ericlagergren ericlagergren commented Feb 12, 2019

I think there's more overlap than it would seem. A 128-bit decimal gives you 34 significant digits which is plenty for most regular usage, like wanting to perform accurate monetary arithmetic.

And, as Brad's link mentioned, four of the six listed languages have only big decimals. So, I think it's somewhat germane :)

A point to consider (though I'm definitely in support of adding decimal type(s)): if decimals are added there will likely need to be support in the math/ package, like math and math/cmplx.

@djadala
Copy link
Contributor

@djadala djadala commented Feb 13, 2019

In this proposal, i expect only fixed size decimals, and i need them for financial computations.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Feb 13, 2019

For people like me who know nothing, can you expand on why decimal floats are better for financial computations that 1) ordinary floats; 2) the int64 type? Just pointing to a paper would be great. Thanks.

@deanveloper
Copy link

@deanveloper deanveloper commented Feb 13, 2019

Section 1.1 discusses a few reasons here - http://speleotrove.com/decimal/IEEE-cowlishaw-arith16.pdf

@ericlagergren
Copy link
Contributor

@ericlagergren ericlagergren commented Feb 13, 2019

That PDF is good. His FAQ sections is good as well: http://speleotrove.com/decimal/decifaq1.html#inexact

TL;DR: floats are inaccurate because they can't exactly store base 10 numbers.

Integers (representing the smallest currency amount, e.g. 1 cent USD) work well, but can be annoying to use for anything other than basic arithmetic.

And for non-monetary calculations integers don't work. E.g., http://speleotrove.com/decimal/decifaq4.html#order

@djadala
Copy link
Contributor

@djadala djadala commented Feb 13, 2019

In my company, we used all 3 alternatives:
decimal in c/c++
int64 in go, c/c++,
float in c/c++, perl

floats are not exact numbers, making some elementary calculations can give surprising ( for financial people) results.
just storing and loading float64 from database (oracle use some sort of decimals for numbers) can lead to inexact value, and some financial applications are 90% database io.
int64s mostly works, but are cumbersome (IMO), in all IO to external systems (including databases) you must convert them to "right" form.

hope this is useful

@alanfo
Copy link

@alanfo alanfo commented Jun 15, 2019

As I can personally testify, the 128 bit decimal type is great for writing financial applications in C# which need to be accurate to the cent, penny etc. as you don't have to worry about the rounding errors which can accumulate if you use their float or double types. The results are always exact even for the largest amounts likely to be encountered in practice.

The alternative is to always work in cents using their 64 bit long type and then div/rem by 100 when you need to display or store the final result. However, this is both tedious and error prone and may even not be large enough when you're dealing with some currencies.

The twin drawbacks are the extra memory required and performance - decimal is much slower than double which in turn is slower than long. However, for most financial programmers this is a price worth paying for the convenience they gain from using decimal.

Back in pre-.NET days, VB6 etc. had a 64 bit Currency type which (IIRC) was scaled to 4 d.p. This was more than adequate for applications which didn't need to process large amounts of money and I always thought it was a mistake not to support it in .NET as well.

So, yes, if you'd like Go to appeal to financial programmers, I think that adding language support for fixed point decimal types would certainly be worthwhile and I'd suggest that 64 bit and 128 bit types (perhaps named dec64 and dec128 respectively) would be adequate as I can't see much point nowadays in having a 32 bit type as well.

As far as untyped constants are concerned, I'd suggest that they should work in a similar fashion to what they do now for the other basic numeric types. So literals would accommodate themselves to whatever type was required, there'd be conversions to/from the other numeric types and so on. However, no untyped numeric constant would have a default type of either dec64 or dec128. So for example, we'd have:

    a := 123.45              // a is float64
    b := dec128(123.45)      // b is dec128
    var c = dec64(a)         // c is dec64
    d := b + 6               // d is dec128
    var e dec64 = 678.9      // e is dec64
@typeless
Copy link
Author

@typeless typeless commented Jul 5, 2019

I believe most people can agree that there is a need for decimals. The question is whether the standard library should provide it or whether it should be a primitive type.
The shortcoming of making decimal as a third-party package is that libraries working with it are hard to cooperate. For example, a package provides finance calculations based on a particular decimal type would not be naturally composable with another decimal type belonging to a time series data store. Then, it would create a situation where a financial application might have to include multiple decimal packages, and programmers have to write conversion code between them, which is clumsy and inefficient.

@alanfo
Copy link

@alanfo alanfo commented Jul 10, 2019

As @deanveloper indicated earlier, the lack of operator overloading in Go would also be a problem if decimal support were provided as a library rather than built into the language.

In fact, I suspect most folks would prefer to stick with what they currently do rather than having to deal with named methods for the basic arithmetic operations. Whilst it's true that you have to do this already for 'big' number arithmetic, the difference is that you have no choice if you need to deal with such numbers.

@vitesse-ftian
Copy link

@vitesse-ftian vitesse-ftian commented Jan 17, 2020

Just some background from data/database people's point of view. Decimal64/128 is really a big help. As said in previous posts, dec32 is probably not interesting.

Unfortunately IEEE allow two different formats for decimal, BID and DPD -- I believe one is from Intel and the other from IBM -- therefore, it is necessary to decide on one format and maybe code conversion routines between these two. Both formats have high quality open source implementations.

Decimal64/128 is slower than double, but not too slow, we measured 2 to 3 x compared to double, which is quite impressive. Databases, for example, PostgreSQL, have to implement their own decimal/numerical type. In case of PostgreSQL, it is a variable length data type, supporting greater precision (but, decimal128 is enough for almost everybody), but much slower.

We need a native decimal64/128 type instead of cgo binding to intel/IBM library because cgo cost may just killed it.

@tc-hib
Copy link

@tc-hib tc-hib commented Jan 20, 2020

Here, we're working with a proprietary language that is made for quickly developing business applications.

Today, I've helped a colleague fix a bug that was due to using binary floats instead of a builtin monetary type. The latter is an 80 bits fixed precision number with 6 decimal places, and it has always been sufficient for us.
My colleague only had to replace the type name everywhere and the bug was fixed.
It wouldn't have been that easy if the monetary type didn't have support for arithmetic operators.

I plan to present Go to my team, but I am a bit afraid of their reaction when I tell them how to write calculations with an external lib.

I think any serious business application should use decimal types. Maybe some developers don't realize that until an accountant asks them why their report doesn't show exactly same numbers as his Excel sheet.

A builtin dec128 would completely fulfill our need.
Of course, the ideal solution would be operator methods.

@tc-hib
Copy link

@tc-hib tc-hib commented Feb 21, 2020

I have just stumbled upon this striking example:
https://play.golang.org/p/w-e7BDgo734

Trying to round 35.855 to 2 decimal places gives unexpected result.
Can be embarrassing in a business app.

For people like me who know nothing, can you expand on why decimal floats are better for financial computations that 1) ordinary floats; 2) the int64 type? Just pointing to a paper would be great. Thanks.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Feb 21, 2020

@tc-hib Thanks. I understand that floating point numbers behave in ways that people find surprising (https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html). That's not the question I was trying to ask.

@tc-hib
Copy link

@tc-hib tc-hib commented Feb 21, 2020

Oh, sorry I didn't get it.

Decimal type is a true silver bullet for all of us who don't need to save every cpu cycle, yet have to follow strict business rules. We simply use it everywhere. Once we've done that, it is really hard to go back, so we would always prefer translating our formulas to method calls rather than using ints or floats.

There are many legacy business apps (in COBOL for example, quite old), Excel sheets, etc. using decimal arithmetic as fluently as integers. I think Go would be a good fit for modernizing all those commerce specific apps thanks to its simplicity, but it would require "translating" every calculation carefully.

I believe using integers and floats is trickier than it seems, because we often have varying number of decimal places (3 for purchases, 2 for sales, and some intermediate calculations not rounded, such as taxes and commercial agreements). If we're only adding and multiplying numbers, or dividing by multiples of 5 and 2, users expect exact results. In fact, they expect same results as Excel sheet or pen and paper.

I must admit i don't have much experience in workarounds, because I've always been working with decimal types, and we nearly never use binary floats, as strange as it may seem.

@dunglas
Copy link
Contributor

@dunglas dunglas commented Jul 19, 2020

The upcoming "Structured Field Values for HTTP" RFC will have a decimal type. To support structured headers and trailers properly in the stdlib, support for decimal floating point will be mandatory.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.