Repository navigation
Releases: d-v-b/refkv
Releases · d-v-b/refkv
Release list
v0.2.0
0.2.0 (2026-10-31)
⚠ BREAKING CHANGES
- zarr: archives name numcodecs.zlib for zlib streams and refkv.tiff_lzw for QPTIFF LZW tiles; refkv no longer registers zlib, imagecodecs_lzw, imagecodecs_jpeg, imagecodecs_jpeg2k or imagecodecs_jpegxr (the last three come from imagecodecs-zarr, in the images extra).
- say zarrify for virtualize, in the code, the CLI, the spec and the docs (#112)
- mp4: cut video chunks at keyframes and decode a chunk's planes once, as a stream (#107)
- ir: add the refkv.stream codec, port MP3 onto it, and virtualize MP4 files as the MP4 profile (§18) (#99)
- reader: read http(s) archives with refkv's own HTTP client, and make obstore an extra (#72)
- python: make python/ a self-contained package versioned from git tags (#71)
Features
- cli: add
vzip convert, a Rust port of the converter that writes identical archives (#78) (a6fb670) - demo: add a QuickTime .mov from DANDI 001538 to the MP4 examples (#154) (3fabeb2)
- demo: add examples of LIF, QPTIFF, HDF5, NumPy, MP3 and FLAC files (#160) (099d5b2)
- demo: add NOAA HRRR and GFS forecasts as GRIB examples (#170) (4b3d193)
- demo: link a hosted .vzip archive as a plain vzip: Neuroglancer source (#126) (1f5ffbc)
- demo: make a DANDI behavior video the MP4 example (#152) (2fa1ea1)
- demo: show a Python script that builds the same Zarr hierarchy (#169) (cbe7ad0)
- demo: show an MP4 file's video in Neuroglancer, with a SLEAP example (#108) (463c316)
- dicom: zarrify JPEG Lossless frames as imagecodecs_ljpeg chunks (#163) (97c4ef0)
- flac: zarrify FLAC files as virtual shards of frames, with a shared shard pass (#121) (df14a47)
- grib: zarrify GRIB2 files, each message a chunk decoded by refkv.grib (#166) (0dd9063)
- hdf5: zarrify HDF5 files, NWB first among them, in hdmf-zarr's Zarr v3 form (#143) (8a56953)
- ir: add the refkv.stream codec, port MP3 onto it, and virtualize MP4 files as the MP4 profile (§18) (#99) (8f79104)
- ir: virtualize MP3 files through the Rust IR, as the MP3 profile (§16) (#91) (0b127f3)
- ir: virtualize MRC2014 files through the Rust IR, as the MRC profile (revision 28) (#83) (5b43073)
- ir: virtualize Olympus VSI slides through the Rust IR, as the VSI profile (revision 29) (#87) (8ab760f)
- js: build refkv_embed.js, the virtualizer and FITS tile decoder for a host that reads the input itself (#109) (62af0f1)
- lif: zarrify Leica LIF files, as the LIF profile (#118) (4a14bec)
- mp4: cut video chunks at keyframes and decode a chunk's planes once, as a stream (#107) (fd78ebe)
- mp4: zarrify H.264 tracks whose pictures are presented before their sync sample (#110) (8fec438)
- mp4: zarrify HEVC video tracks, 8- and 10-bit 4:2:0 (#151) (99eda76)
- ndpi: zarrify JPEG XR levels, with the pixels their streams hold (#125) (3e3cdc1)
- npy: present the deflated members of .npz archives, as np.savez_compressed writes them (#158) (9d409d9)
- npy: zarrify NumPy .npy files and the stored arrays of .npz archives (#156) (2780e2e)
- nrrd: zarrify NRRD files, attached or detached, raw or gzip, as the NRRD profile (#119) (465d575)
- phkv: add the perfect-hash binding (Part 3, draft), standalone and hybrid with vzip (#79) (6e7776e)
- python: build free-threaded wheels for CPython 3.14t and 3.15t (#174) (1f5b7d1)
- reader: read http(s) archives with refkv's own HTTP client, and make obstore an extra (#72) (a165682)
- tiff: zarrify images in strips and LZW, as the QPTIFF profile does (#137) (3b4cbd8)
- tiff: zarrify images with a horizontal predictor through imagecodecs_delta (#139) (242ae27)
- types: type-check the repository with pyrefly's strict preset (#116) (eefa6bd)
- virtualize: add an omero block to TIFF and NDPI images, with channel labels and colors (#95) (7d051e3)
- virtualize: add the QPTIFF profile, with LZW tiles and strips (#84) (850969d)
- virtualize: sample the omero windows of TIFF and QPTIFF images from the coarsest level (#98) (01c4200)
- vsi: zarrify lossless JPEG tiles, and ETS headers that list no dimension sizes (#162) (52304c5)
- zarr: expose refkv's codecs to zarr through entry points, and rename the clashing ones (#136) (1bdab30)
- zarrify: add zarrify_async and zarrify_store_async, which keep the event loop free (#175) (ca01733)
- zarrify: name an input's profile with a source or store class, and choose its layouts (#114) (db24287)
- zarrify: pad NIfTI, MRC and FITS edge chunks with one shared zero data source (#120) (b5f0eda)
Bug fixes
- demo: make two-finger scroll step along the data's first non-x/y axis (#168) ([2ff14a3](https://github.com/...
v0.1.0
What's Changed
- feat(spec): define vzip format version 0 (provisional) and converge it through independent implementations by @d-v-b in #1
- feat(web): serve virtualized TIFFs as plain Zarr from the browser by @d-v-b in #2
- chore: license the repository under MIT OR Apache-2.0 by @d-v-b in #6
- docs: credit kerchunk and VirtualiZarr as the basis of vzip by @d-v-b in #7
- docs: link the live demos from the README by @d-v-b in #8
- fix(web): wait for the vzip worker itself before using the demo by @d-v-b in #9
- feat: virtualize Nikon ND2 files and coalesce nearby range reads by @d-v-b in #10
- feat(virtualize): specify virtualization of TIFF, SVS, NDPI and ND2 in VIRTUALIZE.md by @d-v-b in #17
- feat(convert): write Zarr consolidated metadata into the root zarr.json by @d-v-b in #23
- feat(comparison): measure kerchunk JSON, kerchunk Parquet, Icechunk and vzip on one virtual dataset by @d-v-b in #21
- perf(store): reuse HTTP connections for url sources by @d-v-b in #24
- feat(virtualize): virtualize DICOM, NIfTI and Imaris IMS files by @d-v-b in #20
- feat(virtualize): virtualize N5 and Zarr v2 stores as Zarr v3 by @d-v-b in #30
- feat(virtualize): migrate OME-Zarr 0.4 stores to OME-Zarr 0.5 in place by @d-v-b in #31
- perf(virtualize): hold shared JPEG headers in data sources by @d-v-b in #27
- test(virtualize): shrink the store fixtures, and fix two store review findings by @d-v-b in #29
- feat(virtualize)!: specify each format's Zarr layout and source metadata as a Zarr convention by @d-v-b in #33
- docs(virtualize): add a user-story README per format by @d-v-b in #35
- fix(web): open the demo inside a translated image's data by @d-v-b in #36
- perf(virtualize): read store metadata documents concurrently by @d-v-b in #37
- feat(virtualize)!: virtualize TIFF, ND2 and CZI through a Rust IR core, add CZI and SAFE, and default readers to a secure policy by @d-v-b in #40
- refactor!: reorganize the repository by component and language, and merge each format's profile and convention by @d-v-b in #43
- feat(demo): add examples for the new formats, and show Sentinel-2 products on a map by @d-v-b in #44
- feat(demo): group the examples by format, with a badge and a link to each format's spec by @d-v-b in #45
- fix(virtualize)!: take an IMS channel's color from its lookup table by @d-v-b in #46
- feat(demo): show every image of 2 to 16 channels as one layer, however its channels are chunked by @d-v-b in #48
- docs: link every section reference to its heading by @d-v-b in #50
- fix(virtualize): read store inputs and every file input's request under the reader policy by @d-v-b in #41
- fix(test): serve CONNECT tunnels in the store policy test's proxy, and test Node 24 and 26 by @d-v-b in #52
- docs(spec): split the archive spec into the reference value type and its ZIP binding by @d-v-b in #51
- feat(python)!: add ReferenceStore, an in-memory store of reference values saved as a vzip archive, with a virtual-datasets docs page by @d-v-b in #53
- build!: make zarr an optional dependency, behind a vzip.zarr adapter and the vzip[zarr] extra by @d-v-b in #55
- docs(spec): explain where a reference's bytes that no source holds go by @d-v-b in #54
- feat(spec)!: write the ZIP64 end records in every archive and allow large entries (revision 11) by @d-v-b in #5
- feat(ir)!: describe a store's objects as the sources of one IR by @d-v-b in #56
- feat(virtualize)!: virtualize DICOM through the Rust IR, and make its mirror DICOM's source metadata (revision 26) by @d-v-b in #57
- feat(ir)!: virtualize NDPI through the Rust IR, with its IR mirror as source metadata (revision 26) by @d-v-b in #58
- build!: ship the Rust core inside vzip's wheel, and publish wheels to PyPI on a version tag by @d-v-b in #63
- feat(virtualize)!: move NIfTI onto the Rust IR, and make its mirror the source metadata (revision 26) by @d-v-b in #59
- feat(ir)!: virtualize Sentinel-2 SAFE products through the Rust IR, in both forms by @d-v-b in #60
- feat(ir)!: virtualize Imaris IMS through the Rust IR, with the file's mirror as vzip_source (revision 26) by @d-v-b in #62
- feat(ir)!: move the Zarr v2 store profile onto the multi-source Rust IR (VIRTUALIZE revision 26) by @d-v-b in #61
- feat(python)!: return a ReferenceStore from vzip.virtualize.virtualize by @d-v-b in #64
- test: collect python/tests in one pytest run by @d-v-b in #68
- feat(ir)!: virtualize N5 stores through the multi-source Rust IR, with the store's mirror as vzip_source (revision 26) by @d-v-b in #65
- feat(ir)!: move the OME-Zarr store profile onto the Rust IR as a layer on Zarr v2 (VIRTUALIZE revision 26) by @d-v-b in #66
- refactor(virtualize): reject an unrecognized file from the IR sniff, dropping _virtualize_file by @d-v-b in #67
- refactor!: rename vzip to refkv, move the ZIP binding under refkv.vzip, and split its spec (revision 27) by @d-v-b in #69
Full Changelog: https://github.com/d-v-b/refkv/commits/v0.1.0