Release 4.12.0

Changes from 4.11.0 to 4.12.0

This release focuses on efficient remote arrays. Blosc2 containers can now be
read and written through fsspec URLs, while lazy proxies fetch only the required
blocks, overlap requests and reuse validated caches. Caterva2 arrays gain
block-range reads and concurrent chunk writers. UTF-8 FULL-index lookups also
use substantially less memory by bisecting their vocabulary on disk.

Improvements

• New blosc2[fsspec] extra: blosc2.open(), save_array() and save_tensor()
  accept any fsspec URL — s3://,
  gs://, https://, zip:// and chained URLs. open() can download the
  container, keep a validated local copy with cache_storage=, or fetch only
  requested slices with lazy=True and the new blosc2.FsspecNDSource; lazy
  reads can also use a persistent cache. Saves upload one complete object, but
  URL-backed mutable containers are not supported. Protocol drivers and
  credentials remain the caller's responsibility.

• A C2Array can be written to a chunk at a time, which is how several
  processes fill one remote array at once: update_chunk() (and its async
  aupdate_chunk()) posts one compressed chunk into a slot of a pre-sized
  array, and written_chunks() says which slots hold anything yet. The array is
  laid out with blosc2.uninit() and uploaded -- a couple of hundred bytes
  whatever its size -- and each slot is written once: a second write raises
  blosc2.ChunkAlreadyWritten, which is the whole of the coordination between
  writers. Writing into an empty slot appends to the frame and moves no other
  chunk, so a fill is cheap and a concurrent reader's cached offsets stay good.
  Needs a Caterva2 subscriber that serves the endpoint.

• C2Array.stamp, which is what a Proxy checks its cache against, now names
  which array it is as well as whether it has changed. A subscriber writes a
  nonce into a filled array's vlmeta, so a cache is no longer served against a
  different array that came to sit at the same path with the same size and
  mtime; and a complete array — every chunk written, so every further write
  refused — is stamped without its mtime, so a cache of it survives a republish
  or a copy instead of being thrown away. Arrays that were never filled a chunk
  at a time are stamped exactly as before.

• Proxy.fetch() takes a max_concurrency= argument, and reads it from the
  source when the source has one, so blosc2.open(url, lazy=True, max_concurrency=...)
  overlaps its chunk fetches in a thread pool. Ordinary
  slicing benefits, not just the async afetch(). A lazy fsspec proxy defaults
  to 8, matching what afetch() already used for remote sources; pass 1 for a
  protocol with no latency to hide, where the pool costs ~10 µs per chunk and
  saves nothing. Other sources stay serial unless asked, since this is only safe
  for a thread-safe get_chunk.

• blosc2.open(url, lazy=True) fetches blocks rather than whole chunks when
  a slice lands in a small part of a large one. With the partitions
  blosc2.asarray() picks by default a chunk holds a hundred-odd blocks, so a
  point or window read costs about 1% of what it used to: measured against S3,
  5-17x faster on arrays with multi-megabyte chunks and 2-5x on 1 MB ones. It is
  never a loss, because the two thresholds that decide it — a chunk under a
  megabyte is one cheap request anyway, and wanting more than half of a chunk's
  blocks is wanting the chunk — need nothing read to answer. Peak memory drops
  with the traffic, since a fetch in flight is now a block rather than a chunk.
  bench/ndarray/fsspec-block-granularity.py measures both on any array.

• A Proxy over a C2Array now reads only the required compressed blocks from
  file-backed datasets using concurrent, batched HTTP byte ranges. On
  cat2.cloud's kevlar-tomo.b2nd, this reduced a corner slice from 2.723 MB to
  0.031 MB and a ten-chunk slice from 1.01 s to 0.14 s. Computed datasets fall
  back to whole-chunk reads. blosc2.ByteRangeNDSource provides the same frame
  reader to FsspecNDSource, C2NDSource and custom transports, with lazy,
  persistent caching of frame layout metadata.

• DictStore.member_window(key) says where a leaf's frame lies inside a .b2z,
  as (offset, nbytes). A zip store keeps each external leaf uncompressed, so
  those bytes are the frame that leaf would have been written as on its own --
  which lets a reader take the window instead of the leaf: Caterva2 now serves a
  container leaf from it, so a Proxy over @public/tree.b2z/leaf reads blocks
  exactly as it does over a .b2nd (2.8x on a point read, 20x fewer bytes).
  None where there is no window: a directory-backed store, an embedded leaf, a
  C2Array reference, and a member some other tool repacked compressed --
  which is now also left out of the store's keys rather than read as the deflate
  output it is, and said plainly when it is the store's own super-chunk.

• blosc2.Proxy(src, urlpath=..., mode="a") now adopts the cache left by an
  earlier run, including partially fetched chunks. It validates the cache's
  shape, dtype and source stamp, refetching stale data if the remote array was
  replaced and raising for incompatible caches. Caches from pre-release 4.12.0
  builds require one fresh open with mode="w".

• The source protocol moved to its own module, blosc2.proxy_source:
  ProxySource, ProxyNDSource, ByteRangeNDSource, FsspecNDSource and the
  frame reading behind them. They are still blosc2.X and still reachable as
  blosc2.proxy.X, so nothing outside need change; what moved for good are the
  block-granularity knobs, blosc2.proxy_source.BLOCK_MIN_CBYTES and its
  neighbours. This is what lets proxy.py be imported after schunk and
  indexing, rather than being dragged in ahead of them by every module that
  wants a source.

• Querying a utf8() column through its FULL index no longer materializes the
  index vocabulary. The query literal is turned into an alphabetical rank by
  bisecting the vocabulary sidecar instead, so a lookup reads a few blocks
  rather than one fixed-width entry per distinct value. On a 1 Mrow column of
  near-unique free text, the first lookup goes from ~62 ms and 739 MiB of peak
  memory to ~12 ms and 5.5 MiB.