Debit/Credit Balance Scaling

[sachnk]

Hey!

Big fan of the work you guys are doing.

So, I noticed in your docs that TigerBeetle leverages scaling 64-bit unsigned integers. I understand the rationale here, as doing operations on integers is quite fast. I've used similar techniques in the past.

Where I've run into problems with 10 ^ n integer scaling is when n is quite large, which isn't as uncommon as one might think. When n is large, you aren't left with sufficient room in the remaining bytes to represent the whole-number part. In trading systems, for instance, you can easily hit 10 ^ 10 to represent fractional traded prices. In crypto, the problem gets way worse... an extreme example would be a wei, which would need to be scaled by 10 ^ 18.

Concretely, if you need to scale by 10 ^ 10, then your whole-number part is 2 ^ 64 / 10 ^ 10 = 1,844,674,407, which isn't terribly large.

One solution here is to perhaps have generics to represent balance amounts, and then perhaps the type can be chosen via a build flag, so either 64-bit, 128-bit, arbitrary-decimal type, etc. An arbitrary-decimal type would give the most flexibility, with performance perhaps being a trade-off. For the use cases I'm thinking of, I'd happily make that trade-off.

I do think this problem could become a pretty big deal ultimately for various users who aren't just using TigerBeetle to represent fiat currencies in normal payment flows where n tends to be small.

[jorangreef]

Hey @sachnk!

Big fan of the work you guys are doing.

Awesome to hear that, thanks so much.

We've actually been thinking about 128-bit balances recently. This is something we may do. We had considered a build flag but perhaps we can do it without affecting performance.

It would be great to catch up and chat about this. Would you like to join our Slack and we can setup a time?

[PhakornKiong]

Hey!

Big fan of the work you guys are doing.

So, I noticed in your docs that TigerBeetle leverages scaling 64-bit unsigned integers. I understand the rationale here, as doing operations on integers is quite fast. I've used similar techniques in the past.

Where I've run into problems with 10 ^ n integer scaling is when n is quite large, which isn't as uncommon as one might think. When n is large, you aren't left with sufficient room in the remaining bytes to represent the whole-number part. In trading systems, for instance, you can easily hit 10 ^ 10 to represent fractional traded prices. In crypto, the problem gets way worse... an extreme example would be a wei, which would need to be scaled by 10 ^ 18.

Concretely, if you need to scale by 10 ^ 10, then your whole-number part is 2 ^ 64 / 10 ^ 10 = 1,844,674,407, which isn't terribly large.

One solution here is to perhaps have generics to represent balance amounts, and then perhaps the type can be chosen via a build flag, so either 64-bit, 128-bit, arbitrary-decimal type, etc. An arbitrary-decimal type would give the most flexibility, with performance perhaps being a trade-off. For the use cases I'm thinking of, I'd happily make that trade-off.

I do think this problem could become a pretty big deal ultimately for various users who aren't just using TigerBeetle to represent fiat currencies in normal payment flows where n tends to be small.

+1 on this. I think this is an issue when trying to use TigerBeetle with cryptocurrencies. 128-bit seems to be reasonably large enough.

[batiati]

Hi @sachnk and @PhakornKiong, thanks for your input!
We're happy to announce that #1157 just landed with support for 128-bit balances and amounts.

I'm closing this issue as completed, but please feel free to post further comments here or in our Slack channel.