Changes from 4.13.0 to 4.13.1
A maintenance and performance follow-up to 4.13.0 focused on remote data
access and expanded platform support. Lazy access is now the default for remote
arrays and container datasets, local HDF5 files are read directly via h5py
without auxiliary dependencies, warm opens replay cached bootstrap metadata
to eliminate redundant network discovery, disk caches use human-readable
directory structures, and native Windows ARM64 wheels are now built and tested.
Improvements
Remote access
- Lazy access is the default for remote arrays and datasets.
blosc2.open()
on remote.b2ndfiles or container datasets (HDF5, Zarr, B2Z) now returns a
lazyRemoteArraywithout requiring an explicitlazy=True. Dataset paths
require lazy access and rejectlazy=FalsewithNotImplementedErrorrather
than attempting a silent eager download. For standalone array files (such as
.b2nd),cache_dir=keeps the default lazy access and persists fetched chunks
in aRemoteArraycarrier. Passlazy=Falseto download the complete container
there instead.mmap_mode=or a nonzerooffset=forces the eager path. - Direct local HDF5 reads via
h5py. Local.h5/.hdf5datasets are now
read directly throughh5pywithout requiringkerchunk,zarr, or
fsspec. Chunked datasets retain their native HDF5 chunk layout, while
contiguous datasets receive automatically chosen Blosc2 cache chunks. Dataset
attributes are loaded from HDF5 metadata, and file handles are safely closed
when the source object is garbage-collected. Explicitrefs=arguments
continue to select the kerchunk reference reader. - Faster HTTP discovery and instant warm opens.
- Cold HTTP B2Z discovery: Opening a remote B2Z archive over HTTP now
retrieves the ZIP directory tail and remote object identity in a single
bounded range request, halving network round-trips during initial open. - Warm reopens: Reopening cached B2Z, Zarr, or HDF5 sources replays persisted
bootstrap metadata directly from the carrier file, completely bypassing
remote discovery. Sibling HDF5 leaves from the same container under a shared
cache_dir=reuse a single on-disk reference snapshot. - Whole-member prefetch for small B2Z arrays: Members up to 64 KiB are
fetched in full on open (frame header, chunks, and trailing metadata in a
single request). Prefetched chunks live in the standard chunk cache, count
towardcache_bytes, respectmax_cache_bytes, and can be evicted with
trim_cache().
- Cold HTTP B2Z discovery: Opening a remote B2Z archive over HTTP now
Readable disk caches
- Readable source directories for
cache_dir=. Persistent disk caches now
mirror the source filename and internal dataset hierarchy (e.g.,
hierarchy.b2z--03cc6a2f9314/d0/a1.b2nd) with a 12-character identity
fingerprint computed from the URL and storage options. Unsafe characters are
escaped and long components are shortened.RemoteStorecaches follow the
same naming convention. Complete containers localized withlazy=Falseuse
it too (e.g.,temperatures.b2nd--4908c5e5602a/temperatures.b2nd). - Legacy hash-only cache entries are ignored rather than overwritten or deleted;
the first open under 4.13.1 creates a new cache. Explicitcache_path=targets
and direct opens of existing carrier files continue to work. Legacy cache
directories can be safely deleted to reclaim disk space.
Packaging and platform support
- Windows ARM64 builds and wheels. Added native Windows on ARM64
(win_arm64) wheel building and testing via GitHub Actions runners,
including build configuration adjustments for Clang and runtime fallback for
JIT on ARM64. - CI test stability and doctest isolation. Enhanced test suite stability
with logical core utilization in pytest-xdist, per-test workspace isolation
for doctests inconftest.py, and test deadlock diagnostics. - Remote lazy expression examples. Added
examples/remote/lazy-expr.py
and documentation illustrating how to evaluate lazy expressions across
remote B2Z, Zarr, and HDF5 arrays.
Bug fixes
- Embedded frames with an offset. Opening a local file whose name looks
like a container (.h5,.hdf5,.b2z,.zarr) with a nonzerooffset
now opens the embedded Blosc2 frame directly instead of attempting container
parsing. file://HDF5 URLs on Windows. Dataset path separators (::) in
file://URLs survive URL-to-path conversion on Windows platforms
(e.g.,file:///C:/path/x.h5::/d0/a2).- Credential masking.
RemoteArray.infoandstr()mask sensitive query
parameters and credentials in signed URLs.RemoteArray.infonow also works
reliably for local B2Z sources without raising an error. load_tensor()eager access. Explicitly requests eager access so it no
longer inadvertently returns a lazyRemoteArrayintermediate when loading
known remote paths.- Blosc2-filtered HDF5 decoding. Fixed decoding of HDF5 datasets
compressed with the Blosc2 filter (filter ID 32026, as written by
hdf5plugin) when read via kerchunk. Chunks containing multi-chunk
super-chunk frames decoded viablosc2.from_cframe()now handle the resulting
bytes correctly, avoiding anAttributeErroron.tobytes(). - HDF5 reference snapshot publishing. Guarded reference snapshot export so
it is only published when the leaf carrier actually holds a kerchunk map,
fixing a crash when opening local h5py sources with a disk cache on Windows
drive-letter paths. - Read-only portable carriers. Attaching a sparse runtime cache to a
read-only legacy B2Z carrier no longer fails when refreshing bootstrap
metadata; the session safely uses the rebuilt bootstrap in memory.