Survey use cases for BigDecimal (or similar types)

[littledan]

tl;dr: Add your use case in a comment on this issue, even if you don't read the rest of the thread.

Would BigDecimal be useful for you? (Or, would Rational, Decimal128, or some other type be useful for you?) Why? Reports on this thread are encouraged to include:

• Code samples (either current code or ideal/hoped-for code)
• Context of how this shows up in an application and why you need it

Let's just collect use cases in this thread, and then we can consolidate/analyze it all in other issues.

Cases where you're not using a decimal type, and it's causing a bug (with code samples and context/motivation, and how a decimal type could fix it) are also welcome.

[MaxGraey]

I'm also wondering is it necessary implement all methods of Math namespace for BigDecimal?

[littledan]

@MaxGraey Would it be useful for you to do so?

[MaxGraey]

It's hard to tell. Even Julia and R-lang hasn't builtin arbitrary precision decimal arithmetic. Julia has only Rational. And decimal numbers implement as third party libs which it seems not really popular 1 and 2

[littledan]

I guess I am not so surprised if rational took the wind out of decimal's sails in these cases; that matches what I have seen in other languages that include either fixed precision or arbitrary precision decimal: that there isn't a very popular ecosystem library for the other choice. Having a "standard" option is just so useful that it's not worth it to develop or seek out alternatives, even if they may be somewhat technically better for a particular case.

My understanding of the JS ecosystem is that the libraries for decimal are rather popular, and rational libraries are not so popular. Let's discuss the rational alternative in #6 and the JS ecosystem in #22 .

[qzb]

In company I currently work for (@g2a-com), we are dealing with a lot of financial data, which (obviously) cannot be safely represented using floats. We had to solve two problems with representing such data - how to store it in JSON and how to transform it within application.

From my experience currently there are three ways for solving these problems.

1. Integers using number type

Financial amounts can be represented with number type using the lowest subdivision of particular currency (euro cent for euro, kopek for ruble, yen for yen, etc). Such value has to be treated as an integer.

Pros

• Doesn't require additional steps during deserialization.
• Supports operators (+, -, *, /, ===, etc).

Cons

• When comes to representing amounts lesser than lowest subdivision it can be tricky. Obviously developers can use lower denominator, but it falls apart when comes to cryptocurrencies - there is no enough digits of precision to reliably represent them in all cases.
• You have to settle on some minimal precision. In our case this wasn't a problem, we have some some requirements for smallest amounts that can be stored by our systems.
• Developers need to take care getting rid of fraction amounts after divisions.
• Nothing prevents developers to stop treating amount as an "integer".

2. Integers using BigInt

Instead of using number to represent integers, you can use BigInts. By default BigInts aren't JSON serializable, but in this case they should be represented by strings.

Pros

• Supports operators (+, -, *, /, ===, etc).
• Although precision still need to be fixed, but there is not limit for it (so it can be used for cryptocurrencies).
• It makes difficult to use floats for amounts (either by mistake or lack of knowledge).

Cons

• You (still) have to settle on some minimal precision.
• Both serialization and deserailization require additional steps.

3. Strings and decimals

Amounts can be stored in JSON using strings, which can be later wrapped with some decimals library (like decimal.js) for enabling making operations on them.

Pros

• You don't have to worry about precision.
• It makes difficult to use floats for amounts.

Cons

• Additional work during deserialization (all amounts have to be converted to decimals).
• No support for arithmetic operators, all operations have to be done with methods (.plus(), .times(), .equals(), etc).

---

Native Decimal type would be obvious enhancement for 3rd solution, support for arithmetic operators would make working with decimals much easier. AFAIK Decimal128 have sufficient precision to deal with financial amounts (even when cryptocurrencies with come to play). Personally I would prefer to have BigDecimal over Decimal128, but in this case Decimal128 is enough.

I don't see any advantages of using Rational type for storing financial amounts.

[DavidBM]

I'm working at @omnea. We don't execute many monetary operations, but we want to have enough precision in the language to be safe. What we deal with more frequently are annoying issues with the DB and third party apps.

In our case:

MySQL decimal type

We would like to have a type to match the DECIMAL SQL type. One problem we recently hit is: sequelize/sequelize#7465

This forces the libraries to use strings in order to cover all cases. A native type would solve the issues and provide the DB libraries with a type able to match the DB types.

Third party systems

Our system communicates with many third party systems that, for some reason, send long decimals. It would help us if the language has the same capacities as the JSON format ones as it doesn't have a precision limit. Sometimes, parsing a JSON with a number field generated by another language can mean data loss.

Sometimes these decimals are there because other systems are not doing things "the correct way" but at the end we need to deal with it. Almost every other language has bigger floats types and that ends making harder to pass data between systems when one system has no way to parse data from the other system.

In summary, in our case BigDecimal would help us to communicate with other services with less effort. For us is more about having the ability to represent data without loosing precision rather than doing operations with it.

[littledan]

By the way, PRs to the README to add realistic code samples are very much welcome.

[novemberborn]

I work on Ethereum-based financial software. Crypto currencies tend to have many decimal places (ETH has 18 decimal places). We also deal with fiat currencies and convert between crypto and fiat currencies.

It is easiest to think about the various amounts as decimals, rather than big integers, especially when it comes to converting from one to the other.

These amounts are used in multiple programming languages: JavaScript, Golang, Swift, Kotlin… the implementation differences are too subtle for any one person to keep track of. Whatever we do here, I'd prefer if the behavior is predictable and non-surprising.

We need to encode the amounts in JSON (via GraphQL). I'm actually thinking that encoding the mantissa and exponent separately gives us more fidelity than encoding a decimal string, especially if those values are then interpreted by JavaScript, Swift & Kotlin respectively. Having a BigDecimal.from({ mantissa, exponent }) method would be neat.

Most of our arithmetic is addition, subtraction or multiplication. We have exchange rates from crypto currency to fiat, so going in reverse requires division. However we know what precision we'd like to maintain in these scenarios.

The largest integer in Ethereum is a uint256. In practical terms this means the largest decimal we need to be able to represent is 115792089237316195423570985008687907853269984665640564039457584007913129639935. The smallest decimal is 0.000000000000000000000000000000000000000000000000000000000000000000000000000001. decimal128, with 34 significant digits, cannot represent these numbers.

[littledan]

Having a BigDecimal.from({ mantissa, exponent }) method would be neat.

What types would you expect for mantissa and exponent?

The smallest decimal is 0.000000000000000000000000000000000000000000000000000000000000000000000000000001.

Can you say more about how that smallest decimal is represented currently?

[novemberborn]

What types would you expect for mantissa and exponent?

At a minimum, bigint and number (integer) respectively. Though since BigInt('1') and BigInt('0xf') work, it'd be nice if those kinds of strings could be provided as well.

The smallest decimal is 0.000000000000000000000000000000000000000000000000000000000000000000000000000001.

Can you say more about how that smallest decimal is represented currently?

On the wire, we either use decimal strings, or a '1' integer string with an exponent value of 78. There's a different format used in our databases but I'm not up to speed on that one.

[Skalman]

I've created https://baseconvert.com, a simple website supporting conversion between bases. The calculations use 1000 decimal places and an unlimited integer part.
Decimal isn't necessary for this use case, just very high precision.

The code base is quite small, so using a library is okay. Though it'd be nice not to have to rely on a library.

[leebyron]

I support web engineering at @robinhoodmarkets and second @qzb's analysis about numbers in financial applications.

We currently rely on big.js but would love to use standard operators and avoid method chains and conversion functions.

A common issue is accidental coercion. Using object wrappers like Big(), especially those with toJSON or toString methods, can cause operators like + or * to coerce to string or number and behave incorrectly. In cases where this sort of logic is nested underneath other conversions to Big() this can result in functions that execute correctly and even pass TypeScript checks, but produce incorrect results.

Here I'd expect BigDecimal to throw a TypeError when one side of an operator is not also BigDecimal, and that would be that.