Support for bytedelta filter

[FrancescAlted]

This bytedelta filter makes a delta for every byte on a data stream. This is implemented as plugin for Blosc2, and mainly meant to be used after a shuffle filter.

This is essentially based on @aras-p fascinating blog: https://aras-p.info/blog/2023/03/01/Float-Compression-7-More-Filtering-Optimization/, but removing the shuffle part. For speed, it would make sense to intertwine shuffle and bytedelta, as it is done in the blog, but that would require far more work, as Blosc supports shuffle for general (2 to 255) typesizes (channels in the blog jargon).

This is still a bit preliminary (I still need to assess the new dependency on SSE4.1). Some benchmarks should follow soon too.

[FrancescAlted]

Using the latest bytedelta version here, a python script for downloading data and another for transcoding from the ERA5 dataset, I am getting pretty impressive figures.

Using regular shuffle in Blosc2:

Transcoding era5-pds/flux.b2nd (shape: (720, 721, 1440), dtype: float32) to era5-pds/flux-shuffle.b2nd_
  time: 2.23s (1.513 GB/s),	cratio: 5.41x
Transcoding era5-pds/pressure.b2nd (shape: (720, 721, 1440), dtype: float32) to era5-pds/pressure-shuffle.b2nd_
  time: 2.88s (1.173 GB/s),	cratio: 3.60x
Transcoding era5-pds/wind.b2nd (shape: (720, 721, 1440), dtype: float32) to era5-pds/wind-shuffle.b2nd_
  time: 2.16s (1.561 GB/s),	cratio: 6.61x
Transcoding era5-pds/snow.b2nd (shape: (720, 721, 1440), dtype: float32) to era5-pds/snow-shuffle.b2nd_
  time: 0.82s (4.092 GB/s),	cratio: 52.83x
Transcoding era5-pds/precip.b2nd (shape: (720, 721, 1440), dtype: float32) to era5-pds/precip-shuffle.b2nd_
  time: 1.62s (2.087 GB/s),	cratio: 22.57x

Using shuffle + bytedelta:

Transcoding era5-pds/flux.b2nd (shape: (720, 721, 1440), dtype: float32) to era5-pds/flux-bytedelta.b2nd_
  time: 2.30s (1.468 GB/s),	cratio: 6.35x
Transcoding era5-pds/pressure.b2nd (shape: (720, 721, 1440), dtype: float32) to era5-pds/pressure-bytedelta.b2nd_
  time: 3.15s (1.072 GB/s),	cratio: 5.00x
Transcoding era5-pds/wind.b2nd (shape: (720, 721, 1440), dtype: float32) to era5-pds/wind-bytedelta.b2nd_
  time: 2.40s (1.408 GB/s),	cratio: 7.26x
Transcoding era5-pds/snow.b2nd (shape: (720, 721, 1440), dtype: float32) to era5-pds/snow-bytedelta.b2nd_
  time: 0.94s (3.583 GB/s),	cratio: 54.00x
Transcoding era5-pds/precip.b2nd (shape: (720, 721, 1440), dtype: float32) to era5-pds/precip-bytedelta.b2nd_
  time: 1.76s (1.919 GB/s),	cratio: 20.37x

Using bitshuffle:

Transcoding era5-pds/flux.b2nd (shape: (720, 721, 1440), dtype: float32) to era5-pds/flux-bitshuffle.b2nd_
  time: 2.21s (1.525 GB/s),	cratio: 5.76x
Transcoding era5-pds/pressure.b2nd (shape: (720, 721, 1440), dtype: float32) to era5-pds/pressure-bitshuffle.b2nd_
  time: 2.49s (1.353 GB/s),	cratio: 3.98x
Transcoding era5-pds/wind.b2nd (shape: (720, 721, 1440), dtype: float32) to era5-pds/wind-bitshuffle.b2nd_
  time: 2.22s (1.519 GB/s),	cratio: 5.99x
Transcoding era5-pds/snow.b2nd (shape: (720, 721, 1440), dtype: float32) to era5-pds/snow-bitshuffle.b2nd_
  time: 0.94s (3.585 GB/s),	cratio: 45.26x
Transcoding era5-pds/precip.b2nd (shape: (720, 721, 1440), dtype: float32) to era5-pds/precip-bitshuffle.b2nd_
  time: 1.28s (2.638 GB/s),	cratio: 24.35x

All in all, bar the precip dataset, it looks like shuffle + bytedelta is doing a pretty good job in terms of compression ratio. Speed-wise, except for the precip dataset, shuffle + bytedelta seems quite competitive as well. Yay!
Having said that, it would be interesting to analyze why bytedelta does not behave for precip as well as for the others. Food for thought...

[martaiborra]

LGTM!