Default data return format for 4x rewrite

[nazarhussain]

Hi all,

Problem

In Web3.js 1.x releases we are using different data formats for number types data in the different API functions. e.g. web3.eth.getBalance returns number as string. Where as web3.eth.getTransactionCount returns the number as number. So there are different expectations from users when they are expecting numbers data type.

Solution

In 4.x rewrite we make it possible for users to get arbitrary data in different formats. For the data types which do have different structs and primitives available in Javascript. Currently we support following data types and their formats:

Bytes

1. Hex String
2. Buffer
3. Uint8Array

Numbers

1. Number
2. Number String
3. BigInt
4. Hex String

Use can define these formats to specify what format of data they need.

Choices

We are here we are looking for default return formats for the library. We decided hex string for the bytes data, as it's the native format for the Ethereum and compatible with the 1.x.

For numbers data we need to decide which format we should make the default. Each have different prose and cons.

Format	Pros	Cons
Number	1. Native to JS 2. Compatible with arithmetic 3. Higher adoption	1. Can cause overflow.
Number String	1. Easy conversion to native number 2. No overflow error	1. Arithmetic can lead to mis behaviour and can't be detected
Hex String	1. Native to Ethereum clients 2. No overflow error	1. Extra step required for casting 2. Arithmetic can lead to mis behaviour and can't be detected
BigInt	1. Native to JS 2. Compatible with arithmetic 3. Higher adoption 4. No overflow error	1. Requires NodeJS 10+ 2. No IE support

Looking forward to your get your thoughts on what format should be default for the number data type in 4.x release.

[mpetrunic]

Since we plan to add the beacon api which contains values that overflow, number is probably not a viable option.

[davidmurdoch]

BigInt is preferred for all Truffle projects! There is a decimal type now though, so I'm not sure what to do about that.

[filiptronicek]

Just coming in from the talk @nazarhussain just gave at ETH Prague 2022. I would suggest using BigInt, since I believe it is very convenient an easy to use JS-builtins. For support in older environments, we could use something like bigint-polyfill. It uses BigInt if it supported. We could also leverage Google's jsbi, which would be a bit more involving as far as it looks.

[nazarhussain]

@filiptronicek Thanks for your feedback. I also prefer the BigInt for it's usability for intended use case.

Related to polyfills, there can't be a fully compatible one, because JS as language don't support operator overloading. While it's valid expression a + b for native BigInt, it can't be for a polyfill. For JSBI equivalent is to call a function a.add(b), which is not actually a good polyfill should do.

But older platforms here are NodeJS < 10 and IE, I believe no wallet or Daap is developed for these platforms, so it's fine to ignore these.

[jdevcs]

As discussed in web3.js team meeting, and also from feedback in this github discussion using BigInt as default return type for number type values in web3 is more good as there are no overflow errors, and no casting will be required ( in case we use hex extra step of casting is required). It will be breaking change but moving forward with it will be more convenient to use.

[jdevcs]

Marking the discussion as resolved: Consensus around this is to use BigInt as default return type.