Best practices when reading zarr arrays in languages with limited support for data types.

[gdkrmr]

I am working on an R package to read Zarr arrays. What is the best behavior when reading in data types that are not supported by the language?

E.g. R does not have an 64 bit integer, only float64 and int32. Currently I just convert the the 64 bit integer into a 64 bit float. What would be best practice?

• Don't allow reading from non native data types
• Allow, but throw an error when conversion fails.
• Allow, but warn when conversion fails.
• Allow, and turn values that cannot be represented into missings (R has native support for missing values).
• Allow, and turn values that cannot be represented into something else, so they cannot get confused with actual missings.

[meggart]

This has probably affected Zarr.js as well, maybe @gzuidhof can share experiences?

[gzuidhof]

I don't know about best practices, but in the browser we can't read represent numbers larger than ~ 2^53. In some modern browsers (Chrome, FF) its actually possible (see this part of the docs), but I chose not to include support for it by default.

Nobody has asked for it yet, adding support for it would be a matter of a couple of lines here and there, I guess it's a matter of time that we do add it.

I personally think the right thing is to warn when opening a zarr array with int64 type, allow the user to specify what type to downcast to explicitly (float64 or int32), when they do specify it explicitly don't show the warning, and finally throw an error when conversion fails.

[h-mayorquin]

Maybe now it could be implemented now?

https://caniuse.com/?search=BigInt64Array