Skip to content

v0.3.0

Choose a tag to compare

@kevinjohncutler kevinjohncutler released this 28 Sep 04:33
· 88 commits to main since this release

Mostly a speed release, plus JPEG XR support for Zeiss slide scans. There are no breaking changes: nothing public was removed or renamed and no default changed.

Faster

Measured against 0.2.0 on the same inputs, both versions in fresh processes, counted only where both returned identical pixels. Medians on a 20-core Apple silicon Mac and a 64-core x86-64 Linux workstation.

Workload Mac speedup Linux speedup
CZI: build the pyramid reader 51x 35x
CZI: open a slide and read the first crop 6.5x 4.6x
CZI: 50 disjoint crops with read_regions 5.5x 5.9x
CZI: read a 20 x 20 tile mosaic 2.4x 3.0x
CZI: write 8 zstd frames with write_many 4.1x 4.6x
Zarr: whole 4096 x 4096 array, zstd, num_workers=8 3.4x 3.7x
Zarr: same, zlib 3.1x 4.7x
blosc2 from 8 threads, decode / encode 6.5x / 6.6x 6.2x / 7.1x

The full table, with times, is in the README. Zarr regional reads stay serial unless you pass num_workers.

New

  • JPEG XR in CZI. Most Zeiss slide scans store their tiles as JPEG XR (CZI compression 4). A new _jpegxr extension over jxrlib decodes them, and every tile of a 20684 x 32751 Axioscan slide matches czifile exactly. It ships in every wheel, statically linked, so there is nothing extra to install.
  • Parallel Zarr regional reads with num_workers on read_region or OmeZarrArray.
  • CZI: read(out=...) accepts any array including numpy.memmap; read_regions decodes each tile shared by several boxes once, straight into every box; an opt-in decoded-tile cache; write_many compresses on workers while the file stays byte-identical to sequential writes; and opt-in CziWriter(background_encode=True) compresses the current frame while you prepare the next.
  • Caller-owned outputs. Native codecs decode into out= buffers and sinks you provide, seven byte codecs have stateful streaming iterators, and PNG has native row sessions.
  • One bounded pipeline for readers and writers. opencodecs.core.pipeline.map_bounded schedules parallel work in order under a byte budget and a shared worker budget, with per-worker scratch and opt-in bit-exact verification of written segments.

Fixed

  • CZI pyramids on real slides. CziPyramidReader took sub-block positions as level pixels, but Zen stores them in full-resolution slide coordinates, often far from zero and negative, so a real slide reported every level as (N, 0). Levels now count from reader.origin, matching czifile, and positions are divided by each level's scale.
  • CZI mosaic overlaps were composed in directory order. They now compose in mosaic-index order, as libCZI and czifile do; on the corpus slide that changes 8.9 million overlap pixels at level 0.
  • blosc2 encoding under threads selected its compressor through process-global state, so concurrent encodes could use another thread's compressor. Each call now uses its own context. Chunks with item size 1 and shuffle carry slightly different header flags; old and new chunks decode in both versions.

Build

  • Every wheel ships the same 40 compiled extensions: the 39 of 0.2.0 plus _jpegxr.
  • SZ3 3.3.2, since upstream deleted the 3.3.1 tag.

Full detail in CHANGES.rst.