Skip to content

Releases: medh5-project/medh5

medh5 2.1.0

Choose a tag to compare

@github-actions github-actions released this 15 Oct 22:23
9c5e0ec

A minor release, with no new format version. Files are still written at 1.0 or 1.1, and what 2.0.3 wrote, 2.1.0 writes with the same content_id, but for the imports a fix changed (Behaviour changes). The specification changes by errata only, and the diagnostic code table gains a second revision, W915–W919: validate --level strict can fail a file 2.0 passed, and --codes-revision 1 keeps 2.0's verdicts. The Python and Rust APIs are additive; what a caller can notice is a silently wrong answer fixed, or a deprecation that warns until 3.0.

Answers that were silently wrong, and integrity that cannot be talked past. An amendment refuses a file edited outside medh5 rather than attest the edit, and refuses to replace one that changed after it read it; tracking no longer calls a lesion resolved where a follow-up never imaged it; transform_between no longer answers from whichever grid sorted first; two same-named nested RTSTRUCT ROIs no longer import as a hole; and MONAI can batch medh5 items.

Clinical data in and out, training at scale, converters that say what they did. Clinical histories import from MEDS and FHIR, and a clinical sample can be de-identified; a task can be drawn from a cohort and its split, and repinned. Training gains DDP sharding, sliding-window stitching, a MONAI reader and per-class ignore masks, the batch commands gain --workers, and free-threaded CPython 3.14 gets a wheel. The converters gain FRACTIONAL SEG, codes and laterality in SEG, per-slice PET rescale, RTSTRUCT points, nnU-Net test cases and ITK transforms. Every release artifact is attested, and the crates are published through trusted publishing.

Security

  • A name is no longer pseudonymised without a salt. --profile strict replaced a person or organization agent's name with an unsalted hash, which anyone reverses by hashing candidate names; the run is now refused without --salt. An unsalted run that pseudonymised UIDs reports unsalted_pseudonym; medh5 3.0 requires a salt.
  • A tool's timestamps no longer date the visit. Provenance started and ended, and the file's created, were passed over, though a scan run the day of the visit gives its date. --profile strict cuts them to their year (tool_timestamp), and basic reports them once the dates are shifted.
  • The amount of a date shift can be withheld (D14). A record stating date_shift_days lets one known date restore every other. --withhold-shift (withhold_shift=True) leaves it out and records in the de-identification activity that the dates were shifted, with a salted commitment a re-run checks. --profile strict withholds by default; basic still records it and warns (FutureWarning) that medh5 3.0 will withhold it.
  • A task's sources can be confined to one directory. Whoever wrote a manifest chose every path its runner opened, and T301 telling an absent path from an unreadable one answered whether a path exists. --source-root DIR on task preflight, task reconcile and cache validate, and confine=True (the manifest's directory) or confine=DIR in Python, refuse a URI outside the directory, its . and .. resolved as text, without examining the path (T301, naming the URI only). Lexically, so a symbolic link inside the directory still works; in Rust, SourceAccess and the *_in functions taking it.
  • The crates are published through crates.io's trusted publishing. The release held a long-lived CARGO_REGISTRY_TOKEN and verified one crate at a time, so a failure verifying medh5-cli left medh5-sys and medh5 published. Every crate is now verified first, with no token in existence, and the upload uses a short-lived token issued to the release workflow.
  • Every release artifact carries a build attestation. The binaries' archives on the release page had only SHA256SUMS, which says what was uploaded, not who built it; each archive, wheel and sdist now has a signed statement that the release workflow built it from the tagged commit, checked with gh attestation verify FILE --repo medh5-project/medh5.

Added

  • An image or a grid can be removed, and an image's attributes edited, without rewriting the sample. remove_image and remove_grid join remove_annotation and remove_transform; removing an image drops its acquisition entry and its series_uids mentions too. set_image_attrs changes modality, value_type, value_units, channel_names, the rescale, the window, valid_mask or prov in place, None removing one, where correcting a modality used to mean writing the pixels again. Each is checked as add_image checks it, and the commit refuses what still names a removed object (E101, E503) or leaves no image (E201) or grid (E111).
  • add_mask returns the group it wrote, as every other add_* does. It returned None.
  • A missing, unreadable or directory path raises the matching OSError subclass. MEDH5FileNotFoundError, MEDH5PermissionError and MEDH5IsADirectoryError are each a MEDH5FileError and the built-in of their name, and every MEDH5FileError carries errno and filename, so except FileNotFoundError catches a missing sample. str() is unchanged.
  • An HDF5 read failure raises MEDH5IOError. It is an OSError, as the plain OSError it replaces was, and not yet a MEDH5Error. An engine OSError that names an errno now carries it, so it is the subclass of that errno.
  • PatchSampler.draws and iter_patches take grid=, as draw does. A run of draws can now be pinned to one grid, such as one visit's, whether or not an annotation lives there.
  • medh5.geometry.COORD_SYSTEMS, the coordinate systems the converters map. It is ("LPS", "RAS"), the engine's tuple, which the NIfTI, DICOM SEG, RTSTRUCT and MONAI layers now share instead of each spelling it.
  • A displacement field can be stored in float16. add_transform takes field_dtype=: "float32" (the default, as before), "float16", which refuses a component float16 cannot hold, or "auto", which stores float16 when every displacement is finite, within float16's range and below the 64 voxels §10.3 recommends it for, and float32 otherwise. The field takes half the space.
  • The inverse of a chain or a composite is its steps' inverses, in reverse. inverse() of a ChainTransform, or of a stored composite without a usable inverse_id, returns that chain when every step can be inverted, where it returned None, and InverseTransform.can_invert says so. is_invertible stays the file's claim, and the frame graph is unchanged.
  • Each conformance case states its format version, and a score can be bounded by version and by code revision. expected.json gives each case a format_version, the latest version among the codes it expects, which is not always what its file declares. It also gives the suite's codes_revision. medh5 conformance score --format-version 1.0 scores a 1.0-only implementation on the cases addressed to 1.0. --codes-revision 1 scores one written against the first revision of the code table without the codes later revisions allocated. Either way the cases left out are counted as skipped. The same bounds are available as medh5.conformance.score(..., format_version=, codes_revision=) and Rust's score_with with ScoreOptions. Unbounded, a third party's score fell with every case added by an erratum it was never told about.
  • The diagnostic code table is revisioned, and a validator can pin a revision. Each code names the format version whose rules it reports (since) and the revision of the table that allocated it (revision); Code gains both, and medh5.errors.CODES_REVISION is the table's own. medh5 validate --codes-revision N, validate_file(..., codes_revision=N) and the engine's validate_file_with report only the codes revision N allocated, so a --level strict gate does not move when a later revision adds a warning (1.0 §15.1, §15.2, §16). The conformance suite's codes.json carries both members and the table's revision.
  • The conformance suite builds to the same bytes every time. Two publishes a few minutes apart differed in every case file, with the same content_ids: each sample recorded when it was built, and every group HDF5's creation and modification times. The corpus now pins created, and the engine creates every group, the root included, without timestamps, as it already wrote datasets. A mirror can check a published suite by publishing it again and comparing SHA256SUMS. Any file the engine writes differs from another of the same content only in created; no digest or content_id changes.
  • Agreement compares two vocabularies. compare, compare_voxel and compare_instances take map_classes: "id" (the default), "key" to score classes of one key across two inline label sets (a label set by reference is refused, and one key with two codes of one ontology is C102), or an explicit one-to-one map of a's classes to b's. A class b lacks is listed in skipped with its reason, never scored 0, and each result's b_class_ids names the class of b every class was scored against. medh5 agree takes --map-classes id|key and --map-class A=B. In Rust, curation::agreement::compare_with takes CompareOptions and returns a Compared.
  • Objects pair the way you ask. compare and compare_instances take match_by: "auto" (the default), "iou" to pair by where the objects are even where both carry ids, or "instance_id", which refuses two samples because an id names an object only within its own sample (§7.4). medh5 agree takes --match-by auto|iou|instance-id.
  • Tracking says why each absence is, or is not, a finding. Tracking.reason_at(instance_id, timepoint) and reasons(instance_id) give one of medh5.curation.REASONS: located or class_examined where the object is resolved, outside_fov, ignored, unrelated, no_identity or `not_exa...
Read more

medh5 2.0.3

Choose a tag to compare

@github-actions github-actions released this 14 Oct 16:00
9745a2e

A patch release, with no format change. Four fixes found after 2.0.2, and nothing else: the documented API is unchanged, and no content_id, fingerprint or normal form changes. A task manifest no longer reports anything from a source that is not the version it pins, and a pipe given as a source no longer hangs the command reading it; on Windows a program started while a file is open no longer keeps it locked, which made every amendment fail while it ran; a procedure that asserts it drew a mask attests the mask under effective ordering too; and the DICOM extras declare the pydicom floor highdicom requires.

Security

  • A task manifest reads and reports nothing its pins do not name, and a pipe given as a source is refused. Once a source failed its pin (T302), task preflight kept reading the file: T303 printed the subject id it holds, T304 and T305 compared its clock and events with the other sources', --json reported its whole event table, and T302 itself printed the content_id the file stores. A manifest naming a file its author does not hold therefore reported that file's contents. A source off its pin now adds nothing to its subject and T302 names only the pin, in the engine and in build_document_cache; task reconcile refuses such a source (T302) rather than reconciling its events. A named pipe or a device given as a source blocked task preflight, task reconcile and cache validate inside HDF5's open; every open now refuses what is not a regular file. task-cache-1 gains Appendix A (Errata), and SECURITY.md says what running a manifest someone else wrote can open.

Fixed

  • On Windows, a program started while a file is open no longer keeps it open and locked. HDF5's handle was inheritable, so a child process started while a sample was open kept it open and locked after medh5 closed it, and every rename over the file — the last step of every amendment, scrub --apply, recompress and fix — failed "Access is denied" while the child ran. 2.0.2 fixed this on Linux and macOS; the handle is now kept out of children on Windows too. On every platform a file is opened and marked in one hold of HDF5's lock, and the programs medh5 starts itself (the Python commands of the standalone medh5, the interpreter probe) start only outside it, so none can catch a file between the two; on Linux, where a starting child shares every open file for an instant after the parent resumes, medh5 also waits for that instant to pass before another file is closed or opened.
  • The DICOM extras require pydicom 3.0.1. highdicom 0.23 requires it, so the declared 3.0 floor was one no install of medh5[dicomseg] could resolve; medh5[dicom] and medh5[dicomseg] now say pydicom>=3.0.1, and CI installs exactly that floor.

Behaviour changes

  • A procedure's assertion about its own mask attests the mask. 2.0.2 made an asserting version need only be eligible at the cutoff, but when the link's event endpoint was the asserter itself — seg0 describes lesions_tp0, as both shipped examples phrase it — the asserter still had to be admitted, so under order_by = effective a segmentation procedure of unknown time never attested its mask, and roi = eligible_instances fell back to center_fallback. An endpoint that is the asserting version now needs only its eligibility (1.1 §7.3, §9.1 step 5; task-cache-1 §3.5). The conformance suite gains the task fixture valid-self-asserted-mask, whose row states the annotations its slot may read. No fingerprint changes, so a cache built before still validates: rebuild caches whose slots read self-asserted masks.

medh5 2.0.2

Choose a tag to compare

@github-actions github-actions released this 14 Oct 05:01
b66879f

A patch release, with no format change. What the writer stores keeps the content_id 2.0.1 gave it, though a new file may be chunked differently; the Python API gains only attributes and keyword arguments, and the Rust crate keeps its 2.0.1 signatures, but for three result structs that gain a public field and two key arrays that lose entries no key could match (Behaviour changes). The converters, migrate, clinical export -o and task reconcile -o no longer overwrite the files they read, and what gave a silently wrong answer — a transform chain whose steps do not meet, a merged class, a fabricated negative, an identifier a scrub left behind, a validation that passed a file verify failed — is now refused or reported. A hostile sample can no longer crash the process that opens it, make it load a library or exhaust its memory, nor can a hostile feature cache make FeatureCache.get allocate without bound; the torch loaders no longer serve wrong targets, string datasets read, and encoding, transcoding, digests, index building and field sampling release the GIL.

Security

  • A file can no longer crash the process that opens it by nesting groups. Opening a file checked that it was self-contained with HDF5's recursive H5Lvisit, which has no depth limit: 30,000 nested groups crashed the process (SIGSEGV), and 10,000 took a minute. The check now walks the file itself, a group at a time, and a file nested more than 64 groups deep is refused at open (E001).
  • HDF5's filter plugins load only when asked for. A dataset names its filters by number, and HDF5 loaded one it had not registered from the shared libraries in its built-in search path — on Windows %ALLUSERSPROFILE%\hdf5\lib\plugin, which any local user can create — as whoever opened the file. Blosc2 and Zstandard are compiled in, so plugins are now off, and a dataset whose filter is not built in fails to read with HDF5's "filter plugins disabled". HDF5_PLUGIN_PATH loads them as before, and MEDH5_ALLOW_PLUGINS (set, and not 0) from HDF5's standard directory only, never the directory HDF5 was built in; both are read once per process.
  • A chunk can no longer decompress to far more than its dataset's chunk. The Zstandard and Blosc2 filters allocated whatever size a chunk stated: 32 KiB of run-length blocks asked for a gigabyte, and a Blosc2 header can state 2 GB in a few bytes. Both now cap their output at twice the buffer HDF5 hands them plus 64 KiB, refusing a stated size over it before anything is allocated, and the Blosc2 filter checks that a chunk's header parses (vendor/hdf5-blosc2/PATCHES.md, patch 4).
  • A damaged Blosc2 chunk no longer tears down Blosc2's global state. The filter's failure path called blosc2_destroy(), which frees the library's global context and destroys the mutexes a window read on another thread may be using, and a frame of no chunks made it free an uninitialised pointer. Both are fixed (patches 5 and 6), with tests that call the filter as HDF5 does.
  • A rewrite through a symbolic link rewrites the file the link names. amend, scrub --apply, recompress and fix renamed their result over the link, which became an independent copy, and left the linked file as it was — after scrub --apply, the identifying original. The replacement is now written beside the real file and renamed over it, following a chain of links, and the link is kept (§14.4); a link in a sticky, world-writable directory that another user owns is refused, as Linux's fs.protected_symlinks refuses it.
  • A temporary file is no longer one another user can name in advance. to_nifti and to_nnunetv2 wrote each image and its sidecar to .<pid>-<name> beside the target and renamed it into place, so a link planted at that name, by anyone who could write the directory, was followed and the file it named overwritten. The temporary is now created exclusively, under a name with a random part, as are those of a converter's --report and TaskManifest.save.
  • scrub reads every name, and every copy of the document. A second hard link's name, a soft link and its target, a committed datatype's name and attributes, and /meta under a second name or copied elsewhere all scanned clean. Each name and attribute is now scanned, a second name for /meta or a copy of the document is an identity finding that --apply removes, and what the scan cannot decode — strings nested in compound, array or variable-length types, and byte buffers or undecoded types outside the imaging groups — is reported unreadable, with every non-string dataset outside those groups named in not_checked. A file that scanned clean may not now.
  • An amendment keeps no copy of the identifiers it replaced. It copied /meta by path, so a hard-link alias of it such as x_acme/meta kept the old document — the identifiers scrub --pseudonymise-ids had replaced — and remap_frame_uids left every replaced UID in the file's free space, where HDF5 keeps an overwritten value. The commit now re-links each alias to the new document and, after an alias or a remap, rewrites the file from its root.
  • --pseudonymise-ids replaces an old id wherever a string holds it. A record number in a label, a note or an attribute outlived its pseudonym; every string in the file is now swept for each replaced id of four characters or more (a shorter one, such as 1 or CT, would be found in every number and modality).
  • Identifiers the scrub's tables could never match are found. The key tables were stored unnormalised, so nhs_number, birth_date, date_of_birth and every *_uid key never matched, top-level extra and acquisition keys such as PatientID escaped the key rules, and a value that is wholly a date under an unlisted key, a UID inside text, the de-identification record and agents' organization were never examined. Each is now found, and shifted, pseudonymised or removed as its kind is.
  • A hostile feature cache can no longer make FeatureCache.get allocate up to physical memory. An entry's stored dtype and shape are checked against the cache's declaration before its payload is read, and a mismatch is T402, with nothing read.
  • Advisories are checked weekly and before every release. cargo audit over Cargo.lock and pip-audit over the runtime dependency and every extra but dev run every week, on a pull request that changes Cargo.lock or pyproject.toml, and before the release workflow publishes anything. SECURITY.md inventories the C libraries compiled in, which neither tool can see — HDF5 2.2.0, zlib 1.3.2, C-Blosc2 3.3.5, LZ4 1.10.0 and Zstandard 1.5.7 — with a disposition for every advisory against them.
  • CVE-2026-85091, the heap overflow in zlib's gz_vacate, does not affect medh5. The bundled zlib 1.3.2 is in its range, but nothing medh5 compiles in uses zlib's gzip file interface and no gz* function is linked, so the overflow cannot be reached. vex/medh5.openvex.json says so to scanners, and zlib will be bumped once a release fixes it.
  • Release builds use pinned inputs. Rust is pinned in rust-toolchain.toml (1.97.0); the build script pins maturin and cmake and checks rustup-init against a SHA-256 instead of piping a download to a shell, and the manylinux images are pinned by digest.

Fixed

  • A rewrite that fails leaves the file as it was. A commit whose final close failed renamed the file into place anyway; it now fails the commit. A fixed-length string dataset widened to fit a longer value, as scrub --apply may widen one, is swapped in so that a failure at any step leaves the original and no temporary behind.
  • A dataset whose size does not fit a machine word is an error. Its byte count, the product of a shape the file states, wrapped round, or panicked in a debug build.
  • A read-only file can be amended on Windows, and stays read-only. The temporary file was made read-only before it was opened to be flushed, and Windows does not rename over a read-only file; the order is fixed, and a test amends a read-only file on every platform.
  • A program started while a file is open no longer keeps the file locked. HDF5 opens files without close-on-exec, so a child process started while a sample was open, by any thread, inherited the descriptor and the lock HDF5 takes through it: once the sample was closed, opening it again failed "unable to lock file" until the child exited, and a child started during a write held a descriptor that could write the file. The engine now marks each descriptor close-on-exec as it opens the file, on Linux and macOS.
  • W907 no longer advises int16 for a float image holding NaN or infinity. Only the finite values were tested, so an image with NaN outside its field of view was advised into a type that cannot hold it.
  • repr() of a closed Sample or Image says that it is closed. It raised, and debuggers, loggers and test reports all ask for it.
  • A string dataset reads through Dataset.read() and numpy.asarray(ds). Both refused strings, though the reference documents them; a string dataset now reads as an object array of str in its stored shape, 0-d for a scalar, which ds[...] indexes as NumPy does (Behaviour changes).
  • A free-threaded Python turns the GIL back on when it imports medh5._core. PyO3 declares a module free of the GIL unless told otherwise, so a build from source for free-threaded CPython ran the extension, which was never audited for that, without the GIL. The module now declares that it uses the GIL, so such an interpreter re-enables it at import, with its usual warning.
  • The document's live views stay true to the file after a failed edit. An update refused partway hid the members it had written, a nested sort() that failed partway left the list part-sorted for the next edit to write, and a refused slice assignment on a stale list changed it before refusing.
  • **Under closure = "implicit" the reade...
Read more

medh5 2.0.1

Choose a tag to compare

@github-actions github-actions released this 11 Oct 19:59
9cc6574

Documentation only. The code is 2.0.0's; this release carries the new install instructions and the corrected dashes to the pages PyPI, crates.io and the documentation's stable version show.

Added

  • An install page (Install, after Home): one command for most people, then what each extra is for and what it installs, where a wheel exists and what building from source needs, the command line without Python, the Rust crate, the 1.x line, and what to do when something goes wrong.

Changed

  • The README's install section starts from the task: pip install medh5, a table of extras by what they connect to, then the binary and the crate. The platform details moved to the install page, which the documentation's home page and first tutorial now point to.

Fixed

  • A dash renders as a dash. The documentation, the READMEs and this changelog wrote over six hundred dashes as ---, which GitHub, PyPI, crates.io and the site all print as three hyphens; they are — now, and a test keeps them so.

medh5 2.0.0

Choose a tag to compare

@github-actions github-actions released this 11 Oct 17:27
2b306f8

One format engine, three frontends. The format is now implemented once, in Rust: the medh5 crate is the canonical implementation of the specification — data model, HDF5 I/O, validation, chunked access, compression, geometry and transforms, annotations, provenance and integrity. The Python package is a layer over it (medh5._core, built with PyO3), and the command line is a native binary over the same code. Python keeps what is Python's: NumPy at the API, PyTorch and MONAI, and the NIfTI, DICOM, RTSTRUCT and nnU-Net converters.

Format 1.1: clinical context beside the images, and what was known when. The format gains one optional profile, clinical, and nothing else: a sample may carry its subject's laboratory values, reports and their revisions, diagnoses, medications and assessments on one subject clock, each event an immutable version with when it happened and when it became available (MEDH5 1.1). A sample is still written at the lowest version its content needs, so imaging-only data stays 1.0 and every 1.x reader keeps reading it. Training gets a separately versioned companion contract — task manifests and feature caches pinned to the sample versions they read (task-cache-1) — and a dataset that builds cutoff-aware multimodal batches from it.

What a file is. A sample is written at the lowest format version its content needs. Imaging-only samples are still format 1.0: files written by 1.x and 2.0 are the same format, read by both, and a sample's content_id does not depend on which wrote it. A sample carrying the clinical profile is format 1.1; 1.x opens it and reads its imaging, and its validator reports the profile it does not know (E007). The package version is a major one because of the Python API at the HDF5 boundary — the package no longer hands out h5py objects — and the behaviour changes below.

Added

  • The Rust crates. medh5 (the engine), medh5-sys (HDF5 built from source and linked statically, C-Blosc2, and the Blosc2 and Zstandard HDF5 filters — nothing to install) and medh5-cli (the medh5 binary), for Rust 1.85.1 or later. Their version is the package's, from one place: the Cargo workspace.
  • The native command line, from cargo install --locked medh5-cli, the binaries on each GitHub Release, or Homebrew. The medh5 command pip installs runs the same Rust code, so both give the same output and exit codes. migrate is native; the converters (convert …) and the dataloader row of medh5 bench are the Python integrations, which the binary runs through a Python that has the package (MEDH5_PYTHON, else python3, or python on Windows).
  • Wheels for Linux (x86_64 and aarch64, glibc 2.28 or later), macOS 11 or later (x86_64, arm64) and Windows (x64): one abi3 wheel per platform serves CPython 3.10 and later. For MSVC targets .cargo/config.toml gives the C builds -DNDEBUG: cmake-rs drops CMake's release flags under the Visual Studio generator, and HDF5's assertions would abort the process on a damaged file instead of reporting it. A build that never reads that file — cargo install --locked medh5-cli from crates.io — is stopped by medh5-sys when the HDF5 it links was compiled without NDEBUG (read from HDF5's own libhdf5.settings), naming the variable to set — -DNDEBUG, which cl takes as it takes /DNDEBUG and no shell rewrites, where Git Bash turns /DNDEBUG into a path (C:/Program Files/Git/DNDEBUG) and CMake's compiler check fails; MEDH5_SYS_SKIP_NDEBUG_CHECK=1 overrides. CI damages files for every platform's binary, and for such a build, and fails on an abort.
  • THIRD_PARTY_NOTICES: the licences of everything the engine compiles in — HDF5, C-Blosc2 and its codecs, the HDF5-Blosc2 filter, every Rust crate and the Rust standard library — in the wheel's .dist-info/licenses/, the sdist and every binary archive; each crate carries LICENSE. The file is generated from Cargo.lock (.github/scripts/third_party_notices.py), and CI fails while it is stale. The wheels declare the licences of everything they compile in, where 1.x declared MIT, and so does the Homebrew formula: MIT AND BSD-3-Clause AND BSD-2-Clause AND Zlib AND Apache-2.0 AND MIT-0.
  • A release that fails partway can be finished by re-running its failed jobs: each publishing step skips what is already published and identical to the run's build — on PyPI the wheels and the sdist, not the attestations Trusted Publishing writes beside them — publishes the rest, and stops on anything published from something else; a last job checks PyPI, crates.io, the release page and the tap. Each crate is verified from its package without CARGO_REGISTRY_TOKEN in the environment, where a dependency's build script could read it, and published once crates.io's index serves the crate before it. Release notes longer than a GitHub release takes (125,000 characters) are refused before anything is published, not once the packages are on PyPI and crates.io. In this repository a missing CARGO_REGISTRY_TOKEN or HOMEBREW_TAP_TOKEN fails its job instead of warning. CONTRIBUTING.md has the runbook, and what to set up before the first release to each registry. CI's one required check is ci-ok, which needs every other job.
  • The sdist builds with maturin 1.9.3 or later, the first to write PEP 639's License-Expression (CI builds it with exactly 1.9.3), and carries the workflows, release scripts, mkdocs.yml and docs hooks its own tests/project and the documentation read; CI runs those tests from the unpacked archive.
  • CI runs what ships, where it ships: the test suite on the Linux aarch64 and macOS x86_64 wheels too, medh5.torch on Windows, the engine's tests on Windows and macOS, cargo package for the three crates on every pull request, and each binary's hand-over of the converters to Python.
  • Types for the engine: medh5/_core.pyi, checked against the built module by stubtest in the test suite, so mypy --strict users keep the 1.x signatures.
  • medh5.nodes — Group, Dataset and Attrs, the package's views of stored objects (see below); Sample.is_open; medh5.cli.command_tree(), the command line's grammar as data; and an h5py extra (h5py and hdf5plugin) for reading a file with h5py directly.
  • The clinical profile (clinical/meta, events, documents, links): a column layout of primitives — packed UTF-8, int64 time bounds, float64 values, validity masks — written, read, validated and digested by the engine. SampleWriter.set_clock, add_event, add_document, add_link, add_records, drop_clinical; Sample.clinical and Sample.support; the medh5.clinical module (Clock, Event, Document, Link, ClinicalRecords, Selection, SelectionPolicy, select, augment, strip, imaging_events_from_timepoints).
  • Strict prospective selection, normative in 1.1 §9 and implemented once in the engine: only versions known available by the cutoff, the newest of each revision chain, uncertifiable when a later revision's availability is unknown, plans and unknown times excluded, tie groups kept whole, and only the payloads the selected versions attest. latest_provable is the one named alternative.
  • Task manifests (medh5.task/1, medh5.task): sources pinned by content_id (files or collection members), subjects split before rows, modality slots, targets with explicit censoring, task / manifest / row fingerprints, and a preflight that reports every row as eligible, uncertifiable, excluded or in error, with the reason. Fragmented subjects are checked for identity, clock and duplicate agreement, and reconciled.
  • A preflight scales with the cohort, not with rows times history. Subjects are taken one at a time (their files closed before the next's open), each subject's history is prepared for selection once and shared by its rows, each source is walked once and its clinical tables read once — validated, then used — and only events or documents that several fragments hold are digested or compared. The result crosses into Python as columns: Preflight.subjects holds each subject's merged history once (SubjectHistory, EventTable, LinkTable, packed strings), and a RowView is a view over arrays (selected, order, tie_group, plan) whose events, selection and slots are built when read; the columns are built as each subject is handed over (preflight_each), so the cohort is never held as records. On a 60-subject, two-site cohort of 720 rows (bench_preflight.py): 45–61 s and 5.2 GB peak → 3.4 s and 131 MB; the engine alone 10.0 s → about 4 s. At 200 subjects and 4 800 rows: 18.8 s and 481 MB. The command line takes the subjects one at a time too (write_preflight, preflight_each): at 200 subjects its table went from 18.6 s and 742 MiB to 13.5 s and 34 MiB, and --json — 1.6 GB of output, written a subject at a time — from 130 s and 12.3 GB to 70 s and 315 MiB. Validating a cache against a task keeps only what each row admits (Admitted).
  • Feature caches (medh5.cache/1, medh5.cache): event- and patient-level features with a checksummed dependency manifest, validated so that stale (a source changed, T403) is told apart from corrupt (T401, T402: a manifest missing what its level requires, a payload that is not the declared dtype and shape) and from inadmissible (another task, row, cutoff or subject's sources, T404; fitted on another split, T405; encoding versions a row cannot have read, T406). A source pin also requires every dataset the sample's content_id speaks for to carry its digest (T302), so a column added without one is not read under an unchanged pin.
  • medh5.torch.ClinicalTaskDataset and collate_clinical: one item per admitted row — slot windows on ea...
Read more

medh5 1.4.4

Choose a tag to compare

@github-actions github-actions released this 02 Oct 07:43
8530339

Release tooling and README.md only: no format change and no change to the package's behaviour. It exists, like 1.4.3 before it, because README.md is the package's PyPI long description, and the badge links fixed below could reach the project page only through a release.

Added

  • GitHub Releases. release.yml creates one for each tag once PyPI has the distributions: its notes are the version's section of this file, with links into the repository pinned to the tag, and it carries the same wheel and sdist. A tag whose version has no section here now fails before anything is published. Releases for 0.5.0 to 1.4.3 were created from this file.
  • A Read the Docs badge in README.md.
  • README.md's Python examples run in the documentation tests, as every page under docs/ already did. It is the page GitHub and PyPI show, and it was the only one whose examples nothing executed.

Fixed

  • README.md on PyPI. The License and Typed badges were relative links, which resolve on GitHub but 404 on the PyPI project page, where the README is the long description; both are absolute now.
  • README.md's "Write a sample" raised NameError when pasted; it now defines the arrays it writes. "The reasoning behind each" pointed at the documentation's front page rather than the design rationale.

medh5 1.4.3

Choose a tag to compare

@XwK-P XwK-P released this 02 Oct 06:40

Released on 2026-09-27. This GitHub release was added on 2026-10-01.

A documentation release. No format change and no change to the package's behaviour: a review of every page against the code and the specification. It exists, like 1.1.1 before it, because README.md is the package's PyPI long description — the corrected install note and the new links below could only reach the project page through a release.

Removed

  • The design/ records. The 1.0 design proposal and implementation plan were pre-implementation working documents, unmaintained by their own account. The reasoning that still holds — the problems 0.x could not express, the alternatives weighed, the costs accepted, the non-goals — is now a maintained page, Design rationale, and the benchmark results the proposal carried are on Runnable examples beside the scripts that produce them. Old links to the proposal redirect to the new page; the plan remains in the repository's history. Docstrings and medh5 bench --help no longer cite the plan.

Added

  • CONTRIBUTING.md: setup, the gates, how the specification and the tested documentation are kept in step with the code, and the release process.
  • Documentation for what existed but was undocumented: from-nifti --fourth-axis and 4-D series; the grid and roi classification scopes; identity transforms; mask annotations and valid_mask; the Python entry points of every converter; s.index, s.fresh_indices, s.label_set, Timeline.interval_days, commit()/abort(), remap_frame_uids, audit_splits, pack/unpack, grid_patches; the MONAI adapter's space=; PairedPatchDataset's change labels and samples_per_pair.
  • A test holding the conformance page's breakdown of the corpus — valid, invalid, samples, collections, mutated — to the corpus. Only the total was checked, and the breakdown had drifted.

Fixed

  • Specification, editorially. The heading for §5.4 (hierarchy and closure) was missing although three clauses cite it; §6.3's kind registry pointed every geometric kind one subsection early; §4.2 and §8.5 cited §10.3 and §10.5 for §10.4 and §10.6; §2.4's table of /meta members omitted quality. No requirement changed.
  • Claims the code contradicts. dense([65535]) is refused with E404, not answered with an all-zero plane; contours and meshes do not satisfy the det profile; the profile table now states §1.3's requirements; a classification's labels are keyed by class key; Cohort has acquisition_protocol, not protocol; transform_between traverses a link backwards only where an inverse can be evaluated; portable shuffles before gzip; the core install includes jsonschema.
  • Stale numbers. The conformance page's 39 valid cases, 111 samples and 71 mutated cases are 41, 113 and 82; the landing page and the training tutorial quoted foreground-sampling timings from before 1.4.2.
  • The CLI reference now matches the parser: fix --by, track --key, index build defaults, dataset split --assigned-by, dataset stats --set-id and --stride, --report and --json on the converters, and the default annotation ids. medh5 index build --help now says --occupancy is the occupancy map's block size, which is what it always was, not its side length.
  • Page titles that disagreed with the navigation, links labelled with pages that no longer exist, and a tutorial that duplicated the next one.

medh5 1.4.2

Choose a tag to compare

@XwK-P XwK-P released this 02 Oct 06:40
feed0d9

Released on 2026-09-26. This GitHub release was added on 2026-10-01.

The second patch release under the third audit's plan, and the last of it: the cohort, curation and registration tools, performance, and hygiene. No format change — 1.4.2 reads and writes the files 1.0 does, and the conformance corpus is still 117 cases. Most of what changes is a tool answering a question wrongly without saying so: an agreement score for a comparison that measured nothing, a split audit that missed one subject in two groups, a verified file whose attested object had gained an undigested dataset, a box read on another grid at the wrong size. After this release every one of the twelve "final 1.x" properties holds.

Behaviour changes

Read these before upgrading a pipeline. Each is a correction; each can change what existing code produces or accepts:

  • Agreement scores only what was measured (L-29). Object agreement now does what voxel agreement did: an unmatched object whose class the other annotation never examined is left out and listed under skipped — identical work scored 0.667 when the second rater had not been asked to look for a class — while a matched pair always counts. When neither side has anything to compare, value is None rather than 0, mean_iou is None when nothing matched, and to_record() refuses rather than writing a 0 into the file. compare_instances refuses two index annotations on different grids (E101), and boxes in different spaces or in different frames (E414): boxes on a 4 mm and a 1 mm grid were compared in raw index units. compare() picks the comparison the two kinds support and refuses an argument it cannot use; medh5 agree goes through it, so two boxes annotations no longer end in an AttributeError, and --metric or --threshold on the wrong kind is an error rather than ignored. Comparisons across two visits' grids now raise.

  • An agreement record can be stored in the file it describes (L-38). to_record() keyed per_class by class name, and the object record put its mean IoU there as mean_iou; the schema admits only class ids, so every record medh5 produced failed with E005 when written. Records are keyed by class id now, and the object record carries no per_class.

  • Duplicate ids are refused (L-30). A document that declares one agent or activity id twice is refused when read — the reader kept the last, and a no-op amend deleted the other from the file. The validator reports it through the parse (E005; a dedicated code is format 1.1). Two objects sharing an instance_id inside one instances annotation — which §7.4 forbids — are refused by the writer and reported by the validator as E404. Such third-party files no longer open or validate.

  • Split audits find a subject stored under two group_ids (L-33). medh5 splits, audit_splits and the cohort check C202 grouped by the grouping key alone, so one subject in train and in test under two keys reported leaks = 0, ok = True — the case make_splits refuses as C204. Leaks are now found over subjects and keys together (anatomy_units, union–find), and a Leak lists the keys and subjects it joined. Audits can report leaks they used to miss.

  • The frame graph (L-35, S-12). A transform and the stored inverse its inverse_id names are one route: mutually declared inverses — what E505 checks as "mutually consistent" — raised E501 in both directions, and the registration guide told users not to declare them. They now resolve both ways. A composite that contains itself is E501 in the validator and when evaluated, where it validated clean and recursed until RecursionError. transform_between raises KeyError for a key that is not a timepoint, a grid or a frame the file declares; a mistyped "TP1" returned None, the answer for "no registration exists".

  • verify fails a file whose attested object gained an undigested dataset (L-26). content_id covers the digests present, so instance_ids added to a boxes annotation left the file verified and changed tracks(). In a file that declares a content_id, an undigested dataset inside a grid, image, annotation or transform is now listed in VerifyResult.unattested and makes ok false (medh5 verify and recompress print it as UNSIGNED; fix counts it as needing digests). The writer digests every dataset, so none of its files is affected.

  • Index coordinates belong to the annotation's own grid (L-32). as_slices(grid=), to_index(grid=) and to_world(grid=) on an index-space annotation read its coordinates as the other grid's: a box on a 4 mm grid asked for slices on a 1 mm grid of the same frame came back unchanged, covering a quarter of the anatomy. They now convert through world when the frames match and refuse (E414) otherwise; since §3.3 (1.4.1) a grid without a frame_uid relates to nothing, which now applies to world-space conversions onto another grid too. Boxes with slice_index are read on their own grid only.

  • RTSTRUCT rasterisation keeps an island inside a hole (Q-18). Every outer contour on a slice was unioned and every enclosed contour subtracted, so an island drawn inside a hole was erased. A contour's role now follows its nesting depth, and each slice is filled by the even-odd rule the provenance record already named. Rasterised masks gain those islands.

  • recompress --rechunk chunks the way the writer does (L-28) — from each grid's patch_hint or chunk_hint, (1, *spatial) for stacked encodings — where it let h5py guess and spanned the stacked axis the spec keeps at 1, which the validator reported as W902.

  • Sampling-index datasets are compressed (P-12) with the file's label codec, and occupancy is chunked one class per chunk; it was uncompressed and contiguous. Files with an index get smaller.

  • The CLI reports a missing name as an error, not a traceback (Q-12). medh5 convert to-nifti FILE <unknown image> printed a KeyError traceback; any lookup error is now medh5: … and exit 1.

  • SampleWriter.deidentification() with no fields is refused (Q-11). An assert stood there, so under python -O the call wrote nothing and cleared an existing record.

Added

  • medh5.torch.FileGroupedSampler (P-11): shuffles files and yields each file's items together, so an epoch opens each file once per worker. At 100 small files and four items each, 304 opens became 100 and 0.77 ms per item became 0.30. Every dataset gains file_groups(), and the order follows the dataset's set_epoch.
  • medh5.torch.set_cache_size is exported and documented, HandleCache.resize closes idle handles past the new size, and HandleCache.lease(path) holds a handle against eviction.
  • compare(a, b, *, metric=, threshold=, classes=); compare_instances(..., classes=); InstanceAgreement.skipped; VoxelAgreement.class_ids; OBJECT_KINDS.
  • VerifyResult.unattested and Diagnosis.unattested; ATTESTED_GROUPS.
  • Leak.groups, Leak.subjects and medh5.curation.splits.anatomy_units.
  • SamplingIndex.has_class; medh5.storage.chunking.grid_chunks, fit_chunks and field_chunks — the writer's chunk rule, shared with recompress.
  • medh5.transforms.resolve.stored_inverse and medh5.transforms.composite.composite_cycle.
  • medh5 bench gains foreground_sample_many_ms, a foreground draw on a 63-class sample held to the same 1 ms target (T-11); medh5.bench.many_class_measurement and synthetic_many_class_sample.

Fixed

  • The indexed foreground draw is O(1) in the class count too (P-08). SamplingIndex re-read its class table once per class it checked and twice per count it looked up: 190 HDF5 reads and 7.6 ms per draw at 63 classes. It reads them once per open, and a draw reads the one coordinate it picks: one read, 0.10 ms at 63 classes and 0.03 ms at eight (was 0.9). Draws are unchanged for a given seed.
  • Plane counts are bounded (P-09). layers and bitmask counts read a whole plane before slabbing it — 256 MiB for a uint16 layer at 512³, 1 GiB for a bit-plane. The slab reader now descends an axis when one row is over its budget, for every bounded scan.
  • A transcode carries attributes it does not know (Q-13). AnnotationHeader.read dropped them, so re-encoding an annotation lost anything a later minor version had added, which amend carries everywhere else (§16).
  • The handle cache is thread-safe (Q-16). Lookup and eviction are locked, and the datasets hold their handle for the length of an item, so a thread-based loader's eviction can no longer close a file another thread is reading.
  • medh5.monai names the channel axis (Q-17): original_channel_dim comes from the grid, where "no_channel" on an RGB image made EnsureChannelFirst add a second channel axis.
  • medh5 bench --json prints only JSON (Q-19). Its progress lines went to stdout ahead of the document; they go to stderr.
  • recompress of a collection (L-39). A .medh5c shard was rewritten and then raised a KeyError while verifying the output, which it read as a single sample. Each member is verified.

Removed

  • medh5.geometry.grid.iter_spatial_slices and KNOWN_COORD_SYSTEMS, which nothing used (Q-11); the duplicate transforms.apply._refuse_outside.

Specification corrections (Appendix C, entry 21)

  • §10.1 — a transform and the stored inverse its inverse_id names are one route between two frames, traversable both ways; a resolver must not count them as two (S-12).

Conformance, docs, tests and CI

  • The W909-instance-id-two-classes case puts its conflicting id in two annotations — the sample-scoped form Appendix C's §7.4 entry already requires — because inside one annotation a shared id is now E404. Its expected codes are unchanged; the corpus is still 117 cases.
  • A test holds every place the corpus size is written down — README, site, conformance page, Appendix C, CI com...
Read more

medh5 1.4.1

Choose a tag to compare

@XwK-P XwK-P released this 02 Oct 06:40
c92bf42

Released on 2026-09-25. This GitHub release was added on 2026-10-01.

A patch release under the maintenance contract: defect and security fixes, and no format change — 1.4.1 reads and writes the files 1.0 does. A third full audit went after how the files are used: shipped to a collaborator after a scrub, fed to a training loop, recompressed by a curator, amended by a tool one version behind. It found twelve silent wrong answers and one security defect, all fixed here. Four fixes needed a little new API, because the defect cannot be fixed without it — loader item keys, classification rows, drop_identity, --pseudonymise-ids — and each is listed under Added. jsonschema becomes a core dependency. The corpus grows from 116 to 117 cases, and Appendix C now lists twenty corrected clauses.

Behaviour changes

Read these before upgrading a pipeline. Each is a correction; each can change what existing code produces or accepts:

  • Security: files that read outside themselves are refused by every tool (F-22). HDF5 lets a dataset keep its bytes in an arbitrary file on the reader's machine (external storage), and lets a file reach others through external links and virtual datasets. A crafted file whose extension dataset pointed at ~/.ssh/id_rsa validated clean at --level strict, and recompress --out copied the key into the file a curator then shared. open, amend, recompress, repack, pack, verify, validate and the 0.x reader now walk the file first and refuse any such object by name, before reading a byte of it; validate reports E001. Copies no longer expand external data. The writer never produced these objects.

  • scrub examines the whole file and can exit 1 on files that used to pass (F-14, L-36). It kept a list of places to look, and each audit found string-bearing places the list had missed — this time provenance inputs/outputs, the sample's own ids, timepoint text, per-object JSON, and every HDF5 attribute. It now walks every string in /meta, every object name, every attribute and every string dataset (JSON inside a string is decoded and examined as a mapping), skipping only fields named in the code with a reason, and the scan and --apply are one traversal, so "actionable" means what --apply will change. New rules: DICOM UIDs inside provenance references (dicom:<uid>) get the same pseudonym as the field they came from; filesystem paths keep only their file name, or nothing under --profile strict; ages over 90 and organization agents are reported and handled under strict; person names are recognised in any script (Müller^Hans). The DICOM importer records where the ids came from (identity.id_source), scrub reports a sample_id/subject_id copied from PatientID, and a strict --apply exits non-zero while an id is still an identifier — re-mint it, or pass --pseudonymise-ids --salt .... A file named after such an id, or after a UID, is reported too. The attestation's profile string now says what was not examined. After a DICOM import and a strict apply with --pseudonymise-ids, no PatientID, UID or date bytes remain in the file.

  • Loader items carry the ignore region, validity and coverage (F-16). Items gain ignore[ann_id] (the §7.7 region, wherever the encoding keeps it, and any padding), valid[image_id] (the image's valid_mask, never the padding) and meta["annotated"][ann_id] (one flag per label channel). label_format="labelmap" — and medh5.monai.to_dict — now write 65535 at ignored voxels under every encoding, where layers, bitmask, instances and probmap gave 0, and at padding, which was 0 (background). The ignored voxels §7.7 exists to keep out of the loss went into it as negatives.

  • Persistent workers draw new patches every epoch (F-17). The epoch was an int on the dataset; DataLoader(persistent_workers=True) — the documented setup — kept the copy it started with, so every epoch repeated epoch 0's patches. It now lives in shared memory and set_epoch reaches every worker, under fork and spawn. ds.epoch still reads and assigns as before.

  • Two grids without a frame_uid are not registered (F-18). PairedPatchDataset(align= "transform") compared the two frames with ==, and None == None let every unregistered longitudinal sample from the NIfTI and nnU-Net importers through as aligned. It now requires a shared frame (§3.3) or a transform; such samples raise and say to use align="none".

  • A gantry-tilted DICOM stack is refused (F-19). It imported as an orthonormal grid, 17 mm off at the last of twenty slices at 20°. The refusal names the tilt and the worst slice's offset; a stack within half a pixel of its axis imports as before, with a slice_alignment decision recorded.

  • amend refuses a future major version and keeps a later minor (F-20). It restamped a 2.0 file 1.0, and a 1.1 file 1.0 while carrying its 1.1 objects. scrub --apply, fix and seg convert go through amend.

  • Cohort class statistics come from voxel annotations, per sample (F-24). A sample-level classification arrived as "examined, 0 voxels", and class_weights() floored it to 1 voxel — making a one-bit diagnosis the heaviest segmentation class. Classification, geometric and mask annotations no longer contribute; present_in/examined_in count samples, not annotations; and class_weights() leaves out classes with no voxels, with a warning naming them. Weights and prevalence change for cohorts with non-voxel annotations.

  • Contradictory demographics split a PatientID (F-25). An anonymiser's constant PatientID merged two patients into one subject with two visits. Studies whose PatientBirthDate or PatientSex disagree now become one sample each, with a guess in the report naming both values. Such imports produce more files. Study-keyed files are named by their whole StudyInstanceUID, where Path.stem used to chop its last component.

  • commit() runs the validator (L-19–L-22). The writer and the validator each kept their own list of rules and drifted apart. commit() now runs the validator's structural and semantic error rules over the finished file and refuses to write one it would reject (warnings are not raised; measured at about 2 ms on a 192×256×256 sample). A displacement field off its grid's lattice is refused at add_transform (E503); grid units outside mm, um, m, px and time_units outside s, ms are refused (E109); and a form="ref" label set is written, where the writer refused what the validator accepts. Files the validator would reject are no longer written.

  • add_segmentation refuses what it used to half-honour (L-23, L-24, L-37). Passing more than one of masks=, probabilities=, instances=, or an encoding= the argument cannot use, is E404 — masks= was silently dropped beside probabilities=. An ignore region that overlaps a class is stored as the sibling mask under every encoding, so it reads back whole: under labelmap 56 of 64 voxels came back. Such files gain a sibling mask instead of an in-band region. instances=[] with annotated_classes= now writes the "examined, none found" annotation (§7.4) it used to refuse.

  • A transcode from instances to a dense encoding needs drop_identity=True (L-25). Every voxel survived and every instance_id did not; tracks() then found nothing. With the flag (CLI --drop-identity) it proceeds and records a transcode activity saying identity was dropped. Callers that relied on the silent drop must pass the flag.

  • amend keeps a portable file portable (L-27). New datasets were Blosc2, unreadable without hdf5plugin. amend(codec=None) now means the source's codec family.

  • unpack refuses a member name that is a path (L-31). Names read from a shard were used as file names unchecked; ..\..\evil wrote outside the output directory on Windows.

  • Intensity moments honour valid_mask (L-34), and total_voxels counts each measured image's grid once. A CT padded outside its reconstruction circle reported a mean of −1180 where the tissue averages 40. Normalisation statistics change for images that declare a valid mask.

  • A probability map keeps the voxels on its threshold (F-15). Stored as float16, a vote of 1 in 3 became 0.33325 and fell below a threshold of 1/3 — at write time. Readers now compare at the stored precision, and the writer widens to float32 where float16 would change an answer, so such maps take twice the bytes. Existing files read their boundary voxels as contained.

  • The validator no longer invents E411 and W904 on large labelmaps (F-21). It declined to look for an ignore voxel past 64M elements and then reported the uint16 the ignore id requires as an error — medh5 validate failed the writer's own output at ordinary CT sizes.

  • Every encoder refuses a class id outside 1–65534 with E303 (Q-14) before the uint16 cast, where numpy 2 raised OverflowError and numpy 1.24 wrapped the id silently.

  • jsonschema is a core dependency (L-21). Without it E005 went unchecked and commit() wrote documents a machine with it rejects. The schema extra still installs, and adds nothing.

Added

Each is needed by a fix above, and none changes the format:

  • Sample.ignore_region(ann_id, roi=None) and Sample.valid_region(image_id, roi=None) — the two reads the loaders use, public so a custom loader can make them.
  • Loader item keys ignore, valid and meta["annotated"] (F-16).
  • add_classification accepts rows (class, value[, scope_id[, scheme, scheme_value]]), so one class can be asserted per lesion, slice or visit (F-23). The mapping form is unchanged.
  • transcode(..., drop_identity=True), transcode_annotation(..., drop_identity=True) and medh5 seg convert --drop-identity (L-25).
  • scrub.apply(..., pseudonymise_ids=True) and `medh5 scrub --pseudonymise-i...
Read more

medh5 1.4.0

Choose a tag to compare

@XwK-P XwK-P released this 02 Oct 06:40
ea70ce8

Released on 2026-09-04. This GitHub release was added on 2026-10-01.

The final 1.x release. The format version is unchanged: 1.4.0 reads and writes the files 1.0 does. What changes is what the writer does with what it is given. A second full audit went after the inputs rather than the outputs — an ignore region under an encoding chosen by measurement, a provenance id a caller picked, an export of a 2-D sample, a DICOM SEG written the way highdicom writes them by default — and found that the package produced good files while quietly dropping, overwriting or refusing several ordinary things asked of it. Those are the subject of this release. The corpus grows from 115 to 116 cases and Appendix C now lists sixteen corrected clauses.

After this, the package is in maintenance against format 1.0.

Behaviour changes

Read these before upgrading a pipeline. Each is a correction; each can change what existing code produces or accepts:

  • add_segmentation(ignore=...) now keeps the region under every encoding. It was forwarded to the encoder only for labelmap and layers; under bitmask, instances and probmap the array was dropped without a word — no in-band value, no ignore_mask attribute, has_ignore_region reading False. With encoding="auto" the caller could not know which branch they were on, so the same call kept the region for one cohort and lost it for the next, and every ignored voxel became a verified negative for every annotated class with W904 unable to fire. The writer now stores the region where the chosen encoding can hold it: in band for labelmap and layers, and otherwise as a sibling mask annotation named <ann_id>_ignore — same grid, same prov — referenced by the header's ignore_mask (§7.7). Files written with ignore= under those three encodings gain that extra annotation. Passing both ignore= and ignore_mask= is now E404, and encode_voxels, which returns a single payload and so cannot create the sibling, refuses rather than dropping. select_encoding takes ignore= and costs the uint16 widening an in-band ignore forces, so the encoding is chosen with the region in view.

  • Provenance ids are never silently overwritten. Provenance.add_agent and add_activity assigned into a dict, so person("Alice", agent_id="s2") followed by software("tool") — whose automatic id is also s2 — left one agent, the software, and every activity that named Alice named the tool. The file validated, because every reference resolved; it resolved to the wrong node. Both now refuse a duplicate id (replace=True for the one legitimate rewrite), and the writer's automatic ids skip taken ones. Code that relied on overwriting now raises.

  • transcode_annotation(..., "mask") is refused (E404). A mask has no classes, so the conversion OR-ed every class into one volume and left class_ids and annotated_class_ids empty under task="segmentation" — the coverage contract gone, in a file that still validated. TRANSCODABLE is now the whole truth; build a deliberate union with encode_mask and add_mask. The CLI was already safe, since seg convert --to restricts its choices.

  • scrub --apply acts on everything it flags, re-scans what it wrote, and can exit 1. It cleaned extra and acquisition only, then wrote a §11.4 record whose profile string reads "quasi-identifiers removed" and exited 0 — over files still carrying the patient name, institution and referring physician it had itself reported in identity.extra and cohort. Three places were never scanned at all: activity.params (where a converter naturally stores OperatorsName), activity.tool, and quality.issues[].note. All are scanned and cleaned now, along with agent roles and qualifications; under --profile strict a person agent's name becomes a stable pseudonym and flagged free text is removed. apply then re-runs the rules over the file it wrote, records remaining and remaining_actionable in the deidentify activity's params, and the CLI exit code follows that re-scan. identity.sample_id and identity.subject_id are reported and never rewritten — every manifest, split claim and cross-file join names the sample by them.

  • Exported NIfTI files carry their rescale. Image.read() returns stored values, and both exporters wrote them with no scl_slope/scl_inter: a CT imported from DICOM with intercept −1024 left every voxel 1024 HU too high, in to_nnunetv2's only path and in to_nifti(physical=False), with nothing in dataset.json or the report to say so. Both now go through one write_nifti, which records the scale, so every conforming reader — nibabel, SimpleITK, and therefore nnU-Net — sees the physical values. The numbers a downstream tool reads from those files change, which is the correction.

  • 2-D samples can be exported. convert_world was defined for 4×4 affines only, so both exporters stopped on a 2-D grid's 3×3 with "RAS↔LPS conversion is defined for 3-D affines" — and the import → export round trip the converters page promises was broken for every 2-D sample, although both importers accept one. It handles a 2-D affine now, and embed_plane puts it back in the 4×4 a NIfTI file carries (§3.6).

  • A DICOM SEG that omits its empty frames imports. omit_empty_frames=True is highdicom's default and the common form in the wild. The reader assembled a volume from the planes present — two slices, spaced by the distance between them — and the import compared that shape with the grid's and refused with E405 "drawn on a different reconstruction". Frames carry their own ImagePositionPatient, so they are now placed on the grid by position (§3.3), and shape agreement is a question about rows and columns. A frame that genuinely is not a slice of the target grid is still refused, now naming the frame. read_dicom_seg_frames and place_frames are the new public entry points; read_dicom_seg keeps its shape and meaning.

  • recompress verifies its output, and its exit code follows. content_id_preserved compared the attribute the function had just copied with itself — true by construction, including on a file whose bytes were corrupted before the run, which reported "content_id yes", exited 0, and failed verify() immediately after. The re-encoded file is now read back and checked against the digests it carries. It takes roughly twice as long, and can now fail on a file that was already broken.

  • commit(digests=False) keeps content_id. It skipped the root address entirely, so an amend that used the flag for speed turned an addressed file into an unaddressed one. It now stamps the datasets that carry no digest — on an amend, the ones just written — and always computes the root. create(...).commit(digests=False) therefore stamps everything, since nothing has a digest yet.

  • Timepoint.extra, QualityRecord.extra and Activity.extra are gone, and Agent.extra with them. The schema closes all four objects, so anything placed there failed E005 at commit(): an extension point that could not reach a file. An unknown key is now E005 where it is parsed, with the field named. Agent gains the organization field the schema has always had, and Timepoint gains description.

  • open_collection takes no mode. Collection has no mutating method, as medh5.open stopped taking one in 1.3.0.

  • dataset check C202 groups by the §12.2 grouping key, as medh5 splits does, and names the subjects involved. A family or longitudinal group straddling two partitions was a LEAK in one tool and clean in the other, on the same files.

Fixed

  • info, tree, verify, timeline and track accept a .medh5c shard with --key; info and tree without one describe the shard. Six help strings promised collection support the commands did not have.
  • Sample.reference_grid raises E111 on a file with no grids, where it raised a bare IndexError.
  • to_nnunetv2 lists in report.outputs only the files it wrote; it named a labelsTr path whether or not the sample had the annotation.
  • recompress no longer copies the root attributes twice.
  • The annotation reference gave closure as "explicit" | "complete"; the values are explicit and implicit. seg convert is documented with the five targets it accepts.

Performance

  • layers and bitmask cache their class tables per open annotation, as instances has since 1.3.0. dense() asked the table once per class: 63 HDF5 reads of the same table for a 63-class annotation packed into four layers.
  • storage.index._occupancy is vectorised — pad, reshape, any over the block axes — where it was a Python loop over every coarse block: about 80 ms per class at 256³ and 0.6 s at 512³, so build_index on a 200-class annotation spent minutes on a map measured in kilobytes.
  • voxel_counts() answers every class in one pass where the encoding allows it: a slab bincount for labelmap and layers, a per-plane population count for bitmask. It decoded each class separately, and build_index and the unindexed path of dataset stats both go through it.

Specification corrections (Appendix C, entry 16)

  • §7.7 names instances beside bitmask and probmap as an encoding whose ignore region must live in a separate mask annotation. It has no in-band value either, so the clause left a conforming writer with nowhere to put one.

Conformance and tests

  • The corpus grows to 116 cases, with seg-bitmask-ignore-mask: §7.7's separate-mask form as the writer now emits it, partial coverage and W904 silent.
  • tests/v1/test_release_1_4_0.py carries every reproduction above, each named for the finding it came from, and each failing on 1.3.0.
  • tests/v1/test_docs_python.py executes every Python example in the documentation against a real sample. test_docs_examples.py checked the prose and the CLI flags but nev...
Read more