v4.0.0

webknossos

Highlights

• Image conversion rebuilt: TIFF, CZI, DM3/DM4, IMS, MRC, common 2D images are read through dedicated readers that are faster, require less memory, and handle channels, timepoints and extra axes consistently. Zarr, OME-Zarr (incl. .ozx), N5, and Neuroglancer precomputed can now be converted as well.
• Layer.export now writes OME-Zarr, TIFF stacks, and OME-TIFF.
• Remote datasets gain per-mag and per-attachment access modes, which required changing how RemoteDataset is constructed.
• Conversion failures now raise catchable ImageConversionError subclasses instead of generic errors.
• numba and tifffile are no longer installed by default, making the base package roughly 130 MB smaller.
• Bio-Formats support and a number of rarely used formats are gone from the image conversion.

Breaking Changes

• Removed Bio-Formats support, including the use_bioformats argument of Dataset.from_images/add_layer_from_images/RemoteDataset.from_images, the webknossos[bioformats] extra (and its JPype1/JVM dependency), and the DICOM upload example. Formats that were only readable through Bio-Formats — among them .dcm/.dicom, .nd2, .lif, .lsm, .zvi, .nii, .nrrd and .stk — can no longer be converted. Formats with a dedicated reader (TIFF, CZI, DM3/DM4, .ims, MRC, and the common 2D image formats) are unaffected. #1477
• Support for a number of conversion formats has been removed: .jp2/.j2k, .webp, .ppm/.pgm/.pbm, .tga, .sgi, .exr, .hdr, .psd, .dds, .pcx, .ras, .xbm/.xpm, .lsm, .stk, .cine, .seq and .spe. TIFF, CZI, DM3/DM4, .png/.jpg/.jpeg/.gif/.bmp/.ico, .ims and MRC are unaffected. #1498
• tifffile is no longer installed as part of the base package. scm-pims required it unconditionally, so it happened to be present; TIFF conversion now needs the extra it has always been declared under, e.g. pip install "webknossos[tifffile]" or webknossos[all]. #1498
• Removed the timepoint argument of Dataset.add_layer_from_images. Timeseries are now converted into a single layer with a t axis covering all timepoints, so selecting one only discarded data. Remove the argument from calls, and slice the t axis afterwards if you need a single timepoint. #1477
• The arguments of RemoteDataset.__init__ changed. Use RemoteDataset.open instead, as documented. #1492
• Dataset.add_layer_from_images no longer accepts a pims.FramesSequence instance, only paths, glob strings and sequences of paths. #1498
• .czi files with a rotation, illumination, phase, view or block dimension longer than 1 are no longer converted, and now raise UnsupportedImageDataError. Multi-scene .czi files are unaffected. #1498

Added

• The access mode can now be set on the level of mags and attachments, e.g. RemoteLayer.get_mag(mag, access_mode=RemoteAccessMode.PROXY_PATH), RemoteMagView.with_access_mode(), RemoteAttachments.with_access_mode(). This allows remote datasets to be edited (e.g. adding/removing layers/mags) no matter the access mode. The default access mode is inherited from the RemoteDataset. #1492
• Added View.data_format. #1492
• Added support for converting plain Zarr arrays (v2 and v3), OME-Zarr multiscale groups (NGFF 0.4 and 0.5), zipped OME-Zarr archives (.ozx, NGFF RFC-9), plain N5 datasets and N5 multiscale pyramids, and Neuroglancer precomputed volumes via Dataset.from_images/add_layer_from_images. #1495
• Added Layer.export, providing as_ozx (a zipped OME-Zarr archive compliant with NGFF RFC-9), as_tiff_stack (a directory of per-slice TIFF files, one per z-section and per combination of any additional axes) and as_ome_tiff (a single pyramidal OME-TIFF file) to export a layer's data, optionally restricted to a bounding box and/or magnification. All three support layers with additional axes (e.g. time); as_ome_tiff additionally requires the layer's axes to be a subset of c, t, z, y, x. #1502
• Added support for converting Imaris (.ims) files via Dataset.from_images/add_layer_from_images, including multi-channel and multi-timepoint files. Converting a file that is both multi-channel and multi-timepoint yields one layer per channel, each keeping all of its timepoints. #1477
• Added catchable exceptions for the ways image conversion can fail because of its input, so that callers of Dataset.from_images/Dataset.add_layer_from_images no longer have to match on error messages. All of them subclass the new webknossos.ImageConversionError, which subclasses ValueError (what these call sites raised before), and carry the offending path. #1496
  • UnsupportedImageFormatError: no reader handles this format. Also carries the file_extension, the supported_file_extensions, and missing_extras — the extras to install when the format is only unsupported because an optional dependency is missing, e.g. .ims without webknossos[ims].
  • CorruptImageError: the format is supported but the file could not be read, which usually means it is damaged or was uploaded incompletely. Previously a reader-specific error, or an aggregate of all readers' errors. Missing files and permission errors keep raising FileNotFoundError/PermissionError.
  • UnsupportedImageDataError: the images were read, but their data cannot be stored as requested, e.g. a float image converted into a segmentation layer, or extra axes with data_format="wkw". Previously a ValueError or RuntimeError.
• Dataset.add_layer_from_images/from_images now set a default layer color when splitting a multi-channel image into one layer per channel. #1496
• Converting an OME-Zarr file (0.4, 0.5 or .ozx) that carries omero channel metadata now uses it to set each converted layer's default view configuration (color, intensity range, min/max, and whether it starts disabled) and, when channels are split into one layer each, to name the layers from the channel's label instead of channel{N}. #1512
• Added support for WEBKNOSSOS API version 15, which allows segment ids to use the full uint64 range. Such ids may now be serialized as {"customJsonEncoding": "bigint", "value": "<decimal string>"} instead of a plain JSON number, both in API responses and in datasource-properties.json. #1516
• Added neuroglancerPrecomputed as a valid AttachmentDataFormat for MeshAttachments. #1518
• Dataset.from_images/add_layer_from_images now store the value range observed while reading the images as the color layer's default view configuration min/max. Values provided by the source format itself (e.g. an OME-Zarr omero window) take precedence. #1525
• Added RemoteDataset.with_access_mode(), a cheaper alternative to reopen() that reuses the already-fetched properties instead of making a new request. #1492
• Added a gcs extra (pip install "webknossos[gcs]", also part of webknossos[all]) providing gcsfs, which is needed to detect the format of a dataset stored on Google Cloud Storage. #1524
• Added support for passing a dataset id as the first argument of RemoteDataset.open (and Dataset.download, RemoteDataset.trigger_reload_in_datastore), in addition to a dataset name or url. If no dataset with that id exists, the argument is resolved as a dataset name as before. #1527
• Added a mag keyword to Layer.add_mag_as_copy and RemoteLayer.add_mag_as_copy to register the copied data under a different resolution level than it has in the source dataset, matching the existing mag keyword of add_mag_as_ref. #1526

Changed

• Sped up the test suite. #1530
• RemoteDataset.zarr_streaming_path is deprecated. Use the path of an individual mag instead, e.g. layer.get_mag(mag, access_mode=RemoteAccessMode.ZARR_STREAMING).path. #1492
• .czi conversion is faster and uses less memory: only the data needed for each chunk is read, rather than whole image planes. Multi-timepoint .czi files now convert into a single layer with a t axis. #1498
• Converting a file whose format needs an optional dependency now names the extra to install (e.g. webknossos[czi], webknossos[tifffile]) instead of reporting the format as unsupported. #1498
• Dataset.add_layer_from_images accepts any sequence of paths, not only a list. #1498
• Image files matched by a glob pattern or found in a directory are now ordered with natsort, consistently with how Dataset.from_images already sorted them. #1498
• .ims and MRC file conversion now reads shard-sized blocks directly instead of slice-by-slice, improving conversion performance for large files. These formats are always read through a dedicated reader. #1477
• Several channels are now only written into a single layer when they are actually RGB: three uint8 channels of an image format that stores RGB (.png, .jpg, .bmp, …), or, for Dataset.add_layer_from_images without allow_multiple_layers=True, three uint8 channels of any other source too, since WEBKNOSSOS can display them as RGB regardless of format. The channels of every other source — .ims, MRC, TIFF, CZI, DM3/DM4 — are otherwise treated as separate acquisitions and become one layer each, whatever their number and dtype. Dataset.from_images splits them; Dataset.add_layer_from_images raises UnsupportedImageDataError unless allow_multiple_layers=True (write one layer per channel) or channel=<index> (convert one channel) is given, since it adds a single layer by contract. Previously the dtype alone decided, so three 16-bit channels were written into one layer that WEBKNOSSOS cannot display, while an RGB .png was split into three grayscale layers by from_images. #1496
• Dataset.from_images now truncates RGBA to RGB like Dataset.add_layer_from_images does, so a screenshot converts into one RGB layer instead of four grayscale ones. Pass truncate_rgba_to_rgb=False to add_layer_from_images to keep the alpha channel as its own layer. #1496
• Timepoints are no longer split into separate layers by allow_multiple_layers=True; readers that can address them expose all timepoints on a t axis within one layer instead. Readers that cannot (images without dimension metadata) now warn that only the first timepoint is converted. allow_multiple_layers still splits channels. #1477
• Dataset.add_layer_from_images/from_images now verify the dataset properties are internally consistent after converting each layer and raise an error if not, instead of silently continuing. #1477
• Sped up the layer downsampling (median and mode filters), by 3x-6x with less peak memory consumed. #1529
• numba is now an optional dependency and is no longer installed by default, which removes about 130 MB (mostly llvmlite) from the base installation. Install it with pip install webknossos[numba] or pip install webknossos[all]. Without it, the median and mode downsampling works, but is significantly slower. #1529
• Layer.add_mag_as_ref and RemoteLayer.add_mag_as_ref now extend the layer bounding box by the foreign bounding box rescaled to the target mag when the mag keyword overrides the source mag. Previously the unscaled foreign bounding box was used, which did not match where the referenced voxels actually are in Mag(1). #1526

Fixed

• RemoteDataset.reopen now keeps the read_only flag of the original dataset. #1492
• Dataset.from_images no longer raises UnboundLocalError: cannot access local variable 'input_files' when it is given a single file whose format no reader supports; it now reports the format as unsupported like it does for a directory. #1496
• Converting a corrupt MRC file no longer produces an empty layer or fails much later with an unrelated error: mrcfile parses an unreadable header permissively and reports an empty extent, which is now detected up front. #1496
• Dataset.from_images now names the missing optional dependency when the input contains a format whose reader could not be imported (e.g. .ims without webknossos[ims]), instead of only reporting that no supported image data was found. #1477
• Fixed .ims and MRC conversion with mag greater than 1: it crashed with "zero-size array to reduction operation maximum" once the image spanned more than one shard, and wrote corrupted data when converting compressed layers in parallel. #1477
• Fixed .ims and MRC conversion failing with "Could not autodetect how to load a file" when the path was passed as a pathlib.Path rather than a str or UPath. #1477
• Fixed flip_x/flip_y mirroring each shard individually instead of the whole image when converting .ims or MRC files that span more than one shard in x or y, which scrambled the output into mirrored tiles. #1477
• Fixed the bounding box reported for multi-channel n-dimensional images (e.g. a multi-channel, multi-timepoint file) contradicting the layer's channel count, which caused a spurious "Some images are larger than expected" warning and a bounding box that disagreed with the written data. #1477
• Fixed the OME-NGFF axes/coordinateTransformations.scale metadata written for layers with additional axes (e.g. time): it previously always hard-coded 4 entries (c, x, y, z) regardless of the layer's actual dimensionality, producing a spec-invalid document whose axes count didn't match the array's shape. #1502
• Fixed that changing a layer's bounding box resized the arrays of read-only mags, which modified the array metadata of the foreign dataset that a mag added with add_mag_as_ref points to. #1526
• Fixed that local dataset/layer/mag/attachment path resolution on Windows converted mapped/substituted network drives (e.g. Z:\...) to their UNC form (\\server\share\...), which TensorStore's local file driver rejected. #1513
• Fixed that renaming a layer of a zarr-streamed RemoteDataset (where layer metadata cannot be persisted) raised an opaque StopIteration instead of the expected RuntimeError explaining that the layer is read-only. #1518
• Fixed that UnexpectedStatusError and CannotHandleResponseError raised an AttributeError when unpickled (e.g. when raised inside a ProcessPoolExecutor worker), instead of reproducing the original error. #1517
• Fixed that converted datasets always have c,x,y,z axes. #1523
• Fixed RGB TIFFs converting into a single-channel z-stack (the 3 channels read as z-slices) instead of one RGB layer, since tifffile names the samples-per-pixel axis S, not C. A samples axis of exactly 3 tagged photometric=rgb with uint8 data is now recognized as the channel axis, matching how the same pixels convert from a .png. #1499 #1522
• Fixed converting a single 2D multi-channel image (e.g. an RGB .png) at mag greater than 1: it crashed with a mag-alignment AssertionError, since the batch size used to chunk the placeholder bounding box was in mag-native voxels but applied directly to the mag-1 box without scaling by the mag factor. #1522
• Fixed that View.num_channels returned 0 instead of 1 for a view whose bounding box has no channel axis. #1519