Local async benchmarks

[LDeakin]

Depends on zarrs/zarrs_tools#31

These were all run on the benchmark_compress_shard.zarr dataset. This is local filesystem async (not io_uring, blocking under the hood), so not a true async store. However, it shows where the performance limitations are in async zarrs right now.

Entire Array

Command	Timing
zarrs_benchmark_read_async --read-all ...	2641.89ms (8589.93MB decoded @ 3.25GB/s)
zarrs_benchmark_read_sync --read-all ...	1361.56ms (8589.93MB decoded @ 6.31GB/s)
zarrs_benchmark_read_async_as_sync --read-all ...	1573.06ms (8589.93MB decoded @ 5.46GB/s)
zarrs_python_benchmark_read.py --read_all ...	1669.82ms

The first row could be optimised with internal task spawning in async_retrieve_array_subset.

Chunks

Command	Timing
zarrs_benchmark_read_async ...	1246.18ms (8589.93MB decoded @ 6.89GB/s)
zarrs_benchmark_read_sync ...	1177.29ms (8589.93MB decoded @ 7.30GB/s)
zarrs_benchmark_read_async_as_sync ...	1318.43ms (8589.93MB decoded @ 6.52GB/s)
zarrs_python_benchmark_read.py ...	1377.83ms

Inner Chunks

Command	Timing
zarrs_benchmark_read_async --inner-chunks ...	2460.75ms (8589.93MB decoded @ 3.49GB/s)
zarrs_benchmark_read_sync --inner-chunks ...	510.71ms (8589.93MB decoded @ 16.82GB/s)
zarrs_benchmark_read_async_as_sync --inner-chunks ...	2430.88ms (8589.93MB decoded @ 3.53GB/s)
zarrs_python_benchmark_read --inner-chunks ...	???

Async is a lot slower than sync. Probably because the async sharding partial decoder needs optimisation, but could be some async overhead.