Consider writing some CBOR extensions

[dcoutts]

There's a few CBOR spec extensions we might want to consider writing.

The purpose would be to get generic CBOR tools be able to decode some things we might want to use.

• parallel arrays. Values that are logically arrays of records represented as a record of arrays.
• compact single-type primitive arrays. Arrays of e.g. Int/Word16/32/64, Float, Double etc represented not as normal individually CBOR tagged values, but with one single tag up front and then all the values packed together (as a CBOR byte array). Could also include in the array-type tag if it's big or little endian. We'd loose the variable width integers for these but we could do ridiculously fast memcpy (or memcpy + bswap) encoding and decoding. Useful for scientific data.
• bit arrays. This is much like the above, but the encoding has to be slightly different since it has to include a count of the bits.
• multi-dimensional arrays. This is just the array bounds, lower and upper in each dimension, followed by the array data. Could use two tags to cover 0-based and explicit lower bound based.

[dcoutts]

https://tools.ietf.org/html/draft-jroatch-cbor-tags-03

A draft extension for compact single-type primitive arrays! We should implement this and give feedback to the author.

[dcoutts]

Update from the CBOR spec authors is that this extension is likely to become standard (ie end up in the IANA CBOR extension registry), it's just waiting for more feedback from a number of other implementations. Feedback from our impl would also be welcome.

[sopvop]

After 4 months I've found out that I posted comment to wrong issue:

IANA CBOR registry for reference

[dcoutts]

https://datatracker.ietf.org/doc/draft-ietf-cbor-array-tags

The draft standard has been updated to now include fixed tag numbers, so it can actually be implemented now.

It covers arrays of primitives and multi-dimensional arrays. I need to read the bit on column vs row major order more carefully. It might let us represent parallel arrays nicely (i.e. turning a vector of records into a record of vectors, where some/all of these can be primitive arrays).