Repository navigation
Releases: medh5-project/medh5
Release list
medh5 2.1.0
A minor release, with no new format version. Files are still written at 1.0 or 1.1, and what 2.0.3 wrote, 2.1.0 writes with the same content_id, but for the imports a fix changed (Behaviour changes). The specification changes by errata only, and the diagnostic code table gains a second revision, W915–W919: validate --level strict can fail a file 2.0 passed, and --codes-revision 1 keeps 2.0's verdicts. The Python and Rust APIs are additive; what a caller can notice is a silently wrong answer fixed, or a deprecation that warns until 3.0.
Answers that were silently wrong, and integrity that cannot be talked past. An amendment refuses a file edited outside medh5 rather than attest the edit, and refuses to replace one that changed after it read it; tracking no longer calls a lesion resolved where a follow-up never imaged it; transform_between no longer answers from whichever grid sorted first; two same-named nested RTSTRUCT ROIs no longer import as a hole; and MONAI can batch medh5 items.
Clinical data in and out, training at scale, converters that say what they did. Clinical histories import from MEDS and FHIR, and a clinical sample can be de-identified; a task can be drawn from a cohort and its split, and repinned. Training gains DDP sharding, sliding-window stitching, a MONAI reader and per-class ignore masks, the batch commands gain --workers, and free-threaded CPython 3.14 gets a wheel. The converters gain FRACTIONAL SEG, codes and laterality in SEG, per-slice PET rescale, RTSTRUCT points, nnU-Net test cases and ITK transforms. Every release artifact is attested, and the crates are published through trusted publishing.
Security
- A name is no longer pseudonymised without a salt.
--profile strictreplaced a person or organization agent's name with an unsalted hash, which anyone reverses by hashing candidate names; the run is now refused without--salt. An unsalted run that pseudonymised UIDs reportsunsalted_pseudonym; medh5 3.0 requires a salt. - A tool's timestamps no longer date the visit. Provenance
startedandended, and the file'screated, were passed over, though a scan run the day of the visit gives its date.--profile strictcuts them to their year (tool_timestamp), andbasicreports them once the dates are shifted. - The amount of a date shift can be withheld (D14). A record stating
date_shift_dayslets one known date restore every other.--withhold-shift(withhold_shift=True) leaves it out and records in the de-identification activity that the dates were shifted, with a salted commitment a re-run checks.--profile strictwithholds by default;basicstill records it and warns (FutureWarning) that medh5 3.0 will withhold it. - A task's sources can be confined to one directory. Whoever wrote a manifest chose every path its runner opened, and
T301telling an absent path from an unreadable one answered whether a path exists.--source-root DIRontask preflight,task reconcileandcache validate, andconfine=True(the manifest's directory) orconfine=DIRin Python, refuse a URI outside the directory, its.and..resolved as text, without examining the path (T301, naming the URI only). Lexically, so a symbolic link inside the directory still works; in Rust,SourceAccessand the*_infunctions taking it. - The crates are published through crates.io's trusted publishing. The release held a long-lived
CARGO_REGISTRY_TOKENand verified one crate at a time, so a failure verifyingmedh5-clileftmedh5-sysandmedh5published. Every crate is now verified first, with no token in existence, and the upload uses a short-lived token issued to the release workflow. - Every release artifact carries a build attestation. The binaries' archives on the release page had only
SHA256SUMS, which says what was uploaded, not who built it; each archive, wheel and sdist now has a signed statement that the release workflow built it from the tagged commit, checked withgh attestation verify FILE --repo medh5-project/medh5.
Added
- An image or a grid can be removed, and an image's attributes edited, without rewriting the sample.
remove_imageandremove_gridjoinremove_annotationandremove_transform; removing an image drops itsacquisitionentry and itsseries_uidsmentions too.set_image_attrschangesmodality,value_type,value_units,channel_names, the rescale, the window,valid_maskorprovin place,Noneremoving one, where correcting a modality used to mean writing the pixels again. Each is checked asadd_imagechecks it, and the commit refuses what still names a removed object (E101, E503) or leaves no image (E201) or grid (E111). add_maskreturns the group it wrote, as every otheradd_*does. It returnedNone.- A missing, unreadable or directory path raises the matching
OSErrorsubclass.MEDH5FileNotFoundError,MEDH5PermissionErrorandMEDH5IsADirectoryErrorare each aMEDH5FileErrorand the built-in of their name, and everyMEDH5FileErrorcarrieserrnoandfilename, soexcept FileNotFoundErrorcatches a missing sample.str()is unchanged. - An HDF5 read failure raises
MEDH5IOError. It is anOSError, as the plainOSErrorit replaces was, and not yet aMEDH5Error. An engineOSErrorthat names an errno now carries it, so it is the subclass of that errno. PatchSampler.drawsanditer_patchestakegrid=, asdrawdoes. A run of draws can now be pinned to one grid, such as one visit's, whether or not an annotation lives there.medh5.geometry.COORD_SYSTEMS, the coordinate systems the converters map. It is("LPS", "RAS"), the engine's tuple, which the NIfTI, DICOM SEG, RTSTRUCT and MONAI layers now share instead of each spelling it.- A displacement field can be stored in
float16.add_transformtakesfield_dtype=:"float32"(the default, as before),"float16", which refuses a componentfloat16cannot hold, or"auto", which storesfloat16when every displacement is finite, withinfloat16's range and below the 64 voxels §10.3 recommends it for, andfloat32otherwise. The field takes half the space. - The inverse of a chain or a composite is its steps' inverses, in reverse.
inverse()of aChainTransform, or of a stored composite without a usableinverse_id, returns that chain when every step can be inverted, where it returnedNone, andInverseTransform.can_invertsays so.is_invertiblestays the file's claim, and the frame graph is unchanged. - Each conformance case states its format version, and a score can be bounded by version and by code revision.
expected.jsongives each case aformat_version, the latest version among the codes it expects, which is not always what its file declares. It also gives the suite'scodes_revision.medh5 conformance score --format-version 1.0scores a 1.0-only implementation on the cases addressed to 1.0.--codes-revision 1scores one written against the first revision of the code table without the codes later revisions allocated. Either way the cases left out are counted as skipped. The same bounds are available asmedh5.conformance.score(..., format_version=, codes_revision=)and Rust'sscore_withwithScoreOptions. Unbounded, a third party's score fell with every case added by an erratum it was never told about. - The diagnostic code table is revisioned, and a validator can pin a revision. Each code names the format version whose rules it reports (
since) and the revision of the table that allocated it (revision);Codegains both, andmedh5.errors.CODES_REVISIONis the table's own.medh5 validate --codes-revision N,validate_file(..., codes_revision=N)and the engine'svalidate_file_withreport only the codes revisionNallocated, so a--level strictgate does not move when a later revision adds a warning (1.0 §15.1, §15.2, §16). The conformance suite'scodes.jsoncarries both members and the table's revision. - The conformance suite builds to the same bytes every time. Two publishes a few minutes apart differed in every case file, with the same
content_ids: each sample recorded when it was built, and every group HDF5's creation and modification times. The corpus now pinscreated, and the engine creates every group, the root included, without timestamps, as it already wrote datasets. A mirror can check a published suite by publishing it again and comparingSHA256SUMS. Any file the engine writes differs from another of the same content only increated; no digest orcontent_idchanges. - Agreement compares two vocabularies.
compare,compare_voxelandcompare_instancestakemap_classes:"id"(the default),"key"to score classes of one key across two inline label sets (a label set by reference is refused, and one key with two codes of one ontology isC102), or an explicit one-to-one map ofa's classes tob's. A classblacks is listed inskippedwith its reason, never scored 0, and each result'sb_class_idsnames the class ofbevery class was scored against.medh5 agreetakes--map-classes id|keyand--map-class A=B. In Rust,curation::agreement::compare_withtakesCompareOptionsand returns aCompared. - Objects pair the way you ask.
compareandcompare_instancestakematch_by:"auto"(the default),"iou"to pair by where the objects are even where both carry ids, or"instance_id", which refuses two samples because an id names an object only within its own sample (§7.4).medh5 agreetakes--match-by auto|iou|instance-id. - Tracking says why each absence is, or is not, a finding.
Tracking.reason_at(instance_id, timepoint)andreasons(instance_id)give one ofmedh5.curation.REASONS:locatedorclass_examinedwhere the object isresolved,outside_fov,ignored,unrelated,no_identityor `not_exa...
medh5 2.0.3
A patch release, with no format change. Four fixes found after 2.0.2, and nothing else: the documented API is unchanged, and no content_id, fingerprint or normal form changes. A task manifest no longer reports anything from a source that is not the version it pins, and a pipe given as a source no longer hangs the command reading it; on Windows a program started while a file is open no longer keeps it locked, which made every amendment fail while it ran; a procedure that asserts it drew a mask attests the mask under effective ordering too; and the DICOM extras declare the pydicom floor highdicom requires.
Security
- A task manifest reads and reports nothing its pins do not name, and a pipe given as a source is refused. Once a source failed its pin (
T302),task preflightkept reading the file:T303printed the subject id it holds,T304andT305compared its clock and events with the other sources',--jsonreported its whole event table, andT302itself printed thecontent_idthe file stores. A manifest naming a file its author does not hold therefore reported that file's contents. A source off its pin now adds nothing to its subject andT302names only the pin, in the engine and inbuild_document_cache;task reconcilerefuses such a source (T302) rather than reconciling its events. A named pipe or a device given as a source blockedtask preflight,task reconcileandcache validateinside HDF5's open; every open now refuses what is not a regular file. task-cache-1 gains Appendix A (Errata), andSECURITY.mdsays what running a manifest someone else wrote can open.
Fixed
- On Windows, a program started while a file is open no longer keeps it open and locked. HDF5's handle was inheritable, so a child process started while a sample was open kept it open and locked after medh5 closed it, and every rename over the file — the last step of every amendment,
scrub --apply,recompressandfix— failed "Access is denied" while the child ran. 2.0.2 fixed this on Linux and macOS; the handle is now kept out of children on Windows too. On every platform a file is opened and marked in one hold of HDF5's lock, and the programs medh5 starts itself (the Python commands of the standalonemedh5, the interpreter probe) start only outside it, so none can catch a file between the two; on Linux, where a starting child shares every open file for an instant after the parent resumes, medh5 also waits for that instant to pass before another file is closed or opened. - The DICOM extras require pydicom 3.0.1. highdicom 0.23 requires it, so the declared 3.0 floor was one no install of
medh5[dicomseg]could resolve;medh5[dicom]andmedh5[dicomseg]now saypydicom>=3.0.1, and CI installs exactly that floor.
Behaviour changes
- A procedure's assertion about its own mask attests the mask. 2.0.2 made an asserting version need only be eligible at the cutoff, but when the link's event endpoint was the asserter itself —
seg0 describes lesions_tp0, as both shipped examples phrase it — the asserter still had to be admitted, so underorder_by = effectivea segmentation procedure of unknown time never attested its mask, androi = eligible_instancesfell back tocenter_fallback. An endpoint that is the asserting version now needs only its eligibility (1.1 §7.3, §9.1 step 5; task-cache-1 §3.5). The conformance suite gains the task fixturevalid-self-asserted-mask, whose row states the annotations its slot may read. No fingerprint changes, so a cache built before still validates: rebuild caches whose slots read self-asserted masks.
medh5 2.0.2
A patch release, with no format change. What the writer stores keeps the content_id 2.0.1 gave it, though a new file may be chunked differently; the Python API gains only attributes and keyword arguments, and the Rust crate keeps its 2.0.1 signatures, but for three result structs that gain a public field and two key arrays that lose entries no key could match (Behaviour changes). The converters, migrate, clinical export -o and task reconcile -o no longer overwrite the files they read, and what gave a silently wrong answer — a transform chain whose steps do not meet, a merged class, a fabricated negative, an identifier a scrub left behind, a validation that passed a file verify failed — is now refused or reported. A hostile sample can no longer crash the process that opens it, make it load a library or exhaust its memory, nor can a hostile feature cache make FeatureCache.get allocate without bound; the torch loaders no longer serve wrong targets, string datasets read, and encoding, transcoding, digests, index building and field sampling release the GIL.
Security
- A file can no longer crash the process that opens it by nesting groups. Opening a file checked that it was self-contained with HDF5's recursive
H5Lvisit, which has no depth limit: 30,000 nested groups crashed the process (SIGSEGV), and 10,000 took a minute. The check now walks the file itself, a group at a time, and a file nested more than 64 groups deep is refused at open (E001). - HDF5's filter plugins load only when asked for. A dataset names its filters by number, and HDF5 loaded one it had not registered from the shared libraries in its built-in search path — on Windows
%ALLUSERSPROFILE%\hdf5\lib\plugin, which any local user can create — as whoever opened the file. Blosc2 and Zstandard are compiled in, so plugins are now off, and a dataset whose filter is not built in fails to read with HDF5's "filter plugins disabled".HDF5_PLUGIN_PATHloads them as before, andMEDH5_ALLOW_PLUGINS(set, and not0) from HDF5's standard directory only, never the directory HDF5 was built in; both are read once per process. - A chunk can no longer decompress to far more than its dataset's chunk. The Zstandard and Blosc2 filters allocated whatever size a chunk stated: 32 KiB of run-length blocks asked for a gigabyte, and a Blosc2 header can state 2 GB in a few bytes. Both now cap their output at twice the buffer HDF5 hands them plus 64 KiB, refusing a stated size over it before anything is allocated, and the Blosc2 filter checks that a chunk's header parses (
vendor/hdf5-blosc2/PATCHES.md, patch 4). - A damaged Blosc2 chunk no longer tears down Blosc2's global state. The filter's failure path called
blosc2_destroy(), which frees the library's global context and destroys the mutexes a window read on another thread may be using, and a frame of no chunks made it free an uninitialised pointer. Both are fixed (patches 5 and 6), with tests that call the filter as HDF5 does. - A rewrite through a symbolic link rewrites the file the link names.
amend,scrub --apply,recompressandfixrenamed their result over the link, which became an independent copy, and left the linked file as it was — afterscrub --apply, the identifying original. The replacement is now written beside the real file and renamed over it, following a chain of links, and the link is kept (§14.4); a link in a sticky, world-writable directory that another user owns is refused, as Linux'sfs.protected_symlinksrefuses it. - A temporary file is no longer one another user can name in advance.
to_niftiandto_nnunetv2wrote each image and its sidecar to.<pid>-<name>beside the target and renamed it into place, so a link planted at that name, by anyone who could write the directory, was followed and the file it named overwritten. The temporary is now created exclusively, under a name with a random part, as are those of a converter's--reportandTaskManifest.save. scrubreads every name, and every copy of the document. A second hard link's name, a soft link and its target, a committed datatype's name and attributes, and/metaunder a second name or copied elsewhere all scanned clean. Each name and attribute is now scanned, a second name for/metaor a copy of the document is an identity finding that--applyremoves, and what the scan cannot decode — strings nested in compound, array or variable-length types, and byte buffers or undecoded types outside the imaging groups — is reportedunreadable, with every non-string dataset outside those groups named innot_checked. A file that scanned clean may not now.- An amendment keeps no copy of the identifiers it replaced. It copied
/metaby path, so a hard-link alias of it such asx_acme/metakept the old document — the identifiersscrub --pseudonymise-idshad replaced — andremap_frame_uidsleft every replaced UID in the file's free space, where HDF5 keeps an overwritten value. The commit now re-links each alias to the new document and, after an alias or a remap, rewrites the file from its root. --pseudonymise-idsreplaces an old id wherever a string holds it. A record number in a label, a note or an attribute outlived its pseudonym; every string in the file is now swept for each replaced id of four characters or more (a shorter one, such as1orCT, would be found in every number and modality).- Identifiers the scrub's tables could never match are found. The key tables were stored unnormalised, so
nhs_number,birth_date,date_of_birthand every*_uidkey never matched, top-levelextraandacquisitionkeys such asPatientIDescaped the key rules, and a value that is wholly a date under an unlisted key, a UID inside text, the de-identification record and agents'organizationwere never examined. Each is now found, and shifted, pseudonymised or removed as its kind is. - A hostile feature cache can no longer make
FeatureCache.getallocate up to physical memory. An entry's stored dtype and shape are checked against the cache's declaration before its payload is read, and a mismatch isT402, with nothing read. - Advisories are checked weekly and before every release.
cargo auditoverCargo.lockandpip-auditover the runtime dependency and every extra butdevrun every week, on a pull request that changesCargo.lockorpyproject.toml, and before the release workflow publishes anything.SECURITY.mdinventories the C libraries compiled in, which neither tool can see — HDF5 2.2.0, zlib 1.3.2, C-Blosc2 3.3.5, LZ4 1.10.0 and Zstandard 1.5.7 — with a disposition for every advisory against them. - CVE-2026-85091, the heap overflow in zlib's
gz_vacate, does not affect medh5. The bundled zlib 1.3.2 is in its range, but nothing medh5 compiles in uses zlib's gzip file interface and nogz*function is linked, so the overflow cannot be reached.vex/medh5.openvex.jsonsays so to scanners, and zlib will be bumped once a release fixes it. - Release builds use pinned inputs. Rust is pinned in
rust-toolchain.toml(1.97.0); the build script pins maturin and cmake and checksrustup-initagainst a SHA-256 instead of piping a download to a shell, and the manylinux images are pinned by digest.
Fixed
- A rewrite that fails leaves the file as it was. A commit whose final close failed renamed the file into place anyway; it now fails the commit. A fixed-length string dataset widened to fit a longer value, as
scrub --applymay widen one, is swapped in so that a failure at any step leaves the original and no temporary behind. - A dataset whose size does not fit a machine word is an error. Its byte count, the product of a shape the file states, wrapped round, or panicked in a debug build.
- A read-only file can be amended on Windows, and stays read-only. The temporary file was made read-only before it was opened to be flushed, and Windows does not rename over a read-only file; the order is fixed, and a test amends a read-only file on every platform.
- A program started while a file is open no longer keeps the file locked. HDF5 opens files without close-on-exec, so a child process started while a sample was open, by any thread, inherited the descriptor and the lock HDF5 takes through it: once the sample was closed, opening it again failed "unable to lock file" until the child exited, and a child started during a write held a descriptor that could write the file. The engine now marks each descriptor close-on-exec as it opens the file, on Linux and macOS.
W907no longer advisesint16for a float image holding NaN or infinity. Only the finite values were tested, so an image with NaN outside its field of view was advised into a type that cannot hold it.repr()of a closedSampleorImagesays that it is closed. It raised, and debuggers, loggers and test reports all ask for it.- A string dataset reads through
Dataset.read()andnumpy.asarray(ds). Both refused strings, though the reference documents them; a string dataset now reads as an object array ofstrin its stored shape, 0-d for a scalar, whichds[...]indexes as NumPy does (Behaviour changes). - A free-threaded Python turns the GIL back on when it imports
medh5._core. PyO3 declares a module free of the GIL unless told otherwise, so a build from source for free-threaded CPython ran the extension, which was never audited for that, without the GIL. The module now declares that it uses the GIL, so such an interpreter re-enables it at import, with its usual warning. - The document's live views stay true to the file after a failed edit. An
updaterefused partway hid the members it had written, a nestedsort()that failed partway left the list part-sorted for the next edit to write, and a refused slice assignment on a stale list changed it before refusing. - **Under
closure = "implicit"the reade...
medh5 2.0.1
Documentation only. The code is 2.0.0's; this release carries the new install instructions and the corrected dashes to the pages PyPI, crates.io and the documentation's stable version show.
Added
- An install page (Install, after Home): one command for most people, then what each extra is for and what it installs, where a wheel exists and what building from source needs, the command line without Python, the Rust crate, the 1.x line, and what to do when something goes wrong.
Changed
- The README's install section starts from the task:
pip install medh5, a table of extras by what they connect to, then the binary and the crate. The platform details moved to the install page, which the documentation's home page and first tutorial now point to.
Fixed
- A dash renders as a dash. The documentation, the READMEs and this changelog wrote over six hundred dashes as
---, which GitHub, PyPI, crates.io and the site all print as three hyphens; they are—now, and a test keeps them so.
medh5 2.0.0
One format engine, three frontends. The format is now implemented once, in Rust: the medh5 crate is the canonical implementation of the specification — data model, HDF5 I/O, validation, chunked access, compression, geometry and transforms, annotations, provenance and integrity. The Python package is a layer over it (medh5._core, built with PyO3), and the command line is a native binary over the same code. Python keeps what is Python's: NumPy at the API, PyTorch and MONAI, and the NIfTI, DICOM, RTSTRUCT and nnU-Net converters.
Format 1.1: clinical context beside the images, and what was known when. The format gains one optional profile, clinical, and nothing else: a sample may carry its subject's laboratory values, reports and their revisions, diagnoses, medications and assessments on one subject clock, each event an immutable version with when it happened and when it became available (MEDH5 1.1). A sample is still written at the lowest version its content needs, so imaging-only data stays 1.0 and every 1.x reader keeps reading it. Training gets a separately versioned companion contract — task manifests and feature caches pinned to the sample versions they read (task-cache-1) — and a dataset that builds cutoff-aware multimodal batches from it.
What a file is. A sample is written at the lowest format version its content needs. Imaging-only samples are still format 1.0: files written by 1.x and 2.0 are the same format, read by both, and a sample's content_id does not depend on which wrote it. A sample carrying the clinical profile is format 1.1; 1.x opens it and reads its imaging, and its validator reports the profile it does not know (E007). The package version is a major one because of the Python API at the HDF5 boundary — the package no longer hands out h5py objects — and the behaviour changes below.
Added
- The Rust crates.
medh5(the engine),medh5-sys(HDF5 built from source and linked statically, C-Blosc2, and the Blosc2 and Zstandard HDF5 filters — nothing to install) andmedh5-cli(themedh5binary), for Rust 1.85.1 or later. Their version is the package's, from one place: the Cargo workspace. - The native command line, from
cargo install --locked medh5-cli, the binaries on each GitHub Release, or Homebrew. Themedh5commandpipinstalls runs the same Rust code, so both give the same output and exit codes.migrateis native; the converters (convert …) and the dataloader row ofmedh5 benchare the Python integrations, which the binary runs through a Python that has the package (MEDH5_PYTHON, elsepython3, orpythonon Windows). - Wheels for Linux (x86_64 and aarch64, glibc 2.28 or later), macOS 11 or later (x86_64, arm64) and Windows (x64): one
abi3wheel per platform serves CPython 3.10 and later. For MSVC targets.cargo/config.tomlgives the C builds-DNDEBUG: cmake-rs drops CMake's release flags under the Visual Studio generator, and HDF5's assertions would abort the process on a damaged file instead of reporting it. A build that never reads that file —cargo install --locked medh5-clifrom crates.io — is stopped bymedh5-syswhen the HDF5 it links was compiled withoutNDEBUG(read from HDF5's ownlibhdf5.settings), naming the variable to set —-DNDEBUG, whichcltakes as it takes/DNDEBUGand no shell rewrites, where Git Bash turns/DNDEBUGinto a path (C:/Program Files/Git/DNDEBUG) and CMake's compiler check fails;MEDH5_SYS_SKIP_NDEBUG_CHECK=1overrides. CI damages files for every platform's binary, and for such a build, and fails on an abort. THIRD_PARTY_NOTICES: the licences of everything the engine compiles in — HDF5, C-Blosc2 and its codecs, the HDF5-Blosc2 filter, every Rust crate and the Rust standard library — in the wheel's.dist-info/licenses/, the sdist and every binary archive; each crate carriesLICENSE. The file is generated fromCargo.lock(.github/scripts/third_party_notices.py), and CI fails while it is stale. The wheels declare the licences of everything they compile in, where 1.x declaredMIT, and so does the Homebrew formula:MIT AND BSD-3-Clause AND BSD-2-Clause AND Zlib AND Apache-2.0 AND MIT-0.- A release that fails partway can be finished by re-running its failed jobs: each publishing step skips what is already published and identical to the run's build — on PyPI the wheels and the sdist, not the attestations Trusted Publishing writes beside them — publishes the rest, and stops on anything published from something else; a last job checks PyPI, crates.io, the release page and the tap. Each crate is verified from its package without
CARGO_REGISTRY_TOKENin the environment, where a dependency's build script could read it, and published once crates.io's index serves the crate before it. Release notes longer than a GitHub release takes (125,000 characters) are refused before anything is published, not once the packages are on PyPI and crates.io. In this repository a missingCARGO_REGISTRY_TOKENorHOMEBREW_TAP_TOKENfails its job instead of warning.CONTRIBUTING.mdhas the runbook, and what to set up before the first release to each registry. CI's one required check isci-ok, which needs every other job. - The sdist builds with maturin 1.9.3 or later, the first to write PEP 639's
License-Expression(CI builds it with exactly 1.9.3), and carries the workflows, release scripts,mkdocs.ymland docs hooks its owntests/projectand the documentation read; CI runs those tests from the unpacked archive. - CI runs what ships, where it ships: the test suite on the Linux aarch64 and macOS x86_64 wheels too,
medh5.torchon Windows, the engine's tests on Windows and macOS,cargo packagefor the three crates on every pull request, and each binary's hand-over of the converters to Python. - Types for the engine:
medh5/_core.pyi, checked against the built module bystubtestin the test suite, somypy --strictusers keep the 1.x signatures. medh5.nodes—Group,DatasetandAttrs, the package's views of stored objects (see below);Sample.is_open;medh5.cli.command_tree(), the command line's grammar as data; and anh5pyextra (h5pyandhdf5plugin) for reading a file withh5pydirectly.- The
clinicalprofile (clinical/meta,events,documents,links): a column layout of primitives — packed UTF-8,int64time bounds,float64values, validity masks — written, read, validated and digested by the engine.SampleWriter.set_clock,add_event,add_document,add_link,add_records,drop_clinical;Sample.clinicalandSample.support; themedh5.clinicalmodule (Clock,Event,Document,Link,ClinicalRecords,Selection,SelectionPolicy,select,augment,strip,imaging_events_from_timepoints). - Strict prospective selection, normative in 1.1 §9 and implemented once in the engine: only versions known available by the cutoff, the newest of each revision chain, uncertifiable when a later revision's availability is unknown, plans and unknown times excluded, tie groups kept whole, and only the payloads the selected versions attest.
latest_provableis the one named alternative. - Task manifests (
medh5.task/1,medh5.task): sources pinned bycontent_id(files or collection members), subjects split before rows, modality slots, targets with explicit censoring, task / manifest / row fingerprints, and a preflight that reports every row as eligible, uncertifiable, excluded or in error, with the reason. Fragmented subjects are checked for identity, clock and duplicate agreement, and reconciled. - A preflight scales with the cohort, not with rows times history. Subjects are taken one at a time (their files closed before the next's open), each subject's history is prepared for selection once and shared by its rows, each source is walked once and its clinical tables read once — validated, then used — and only events or documents that several fragments hold are digested or compared. The result crosses into Python as columns:
Preflight.subjectsholds each subject's merged history once (SubjectHistory,EventTable,LinkTable, packed strings), and aRowViewis a view over arrays (selected,order,tie_group,plan) whoseevents,selectionandslotsare built when read; the columns are built as each subject is handed over (preflight_each), so the cohort is never held as records. On a 60-subject, two-site cohort of 720 rows (bench_preflight.py): 45–61 s and 5.2 GB peak → 3.4 s and 131 MB; the engine alone 10.0 s → about 4 s. At 200 subjects and 4 800 rows: 18.8 s and 481 MB. The command line takes the subjects one at a time too (write_preflight,preflight_each): at 200 subjects its table went from 18.6 s and 742 MiB to 13.5 s and 34 MiB, and--json— 1.6 GB of output, written a subject at a time — from 130 s and 12.3 GB to 70 s and 315 MiB. Validating a cache against a task keeps only what each row admits (Admitted). - Feature caches (
medh5.cache/1,medh5.cache): event- and patient-level features with a checksummed dependency manifest, validated so that stale (a source changed, T403) is told apart from corrupt (T401, T402: a manifest missing what its level requires, a payload that is not the declared dtype and shape) and from inadmissible (another task, row, cutoff or subject's sources, T404; fitted on another split, T405; encoding versions a row cannot have read, T406). A source pin also requires every dataset the sample'scontent_idspeaks for to carry its digest (T302), so a column added without one is not read under an unchanged pin. medh5.torch.ClinicalTaskDatasetandcollate_clinical: one item per admitted row — slot windows on ea...
medh5 1.4.4
Release tooling and README.md only: no format change and no change to the package's behaviour. It exists, like 1.4.3 before it, because README.md is the package's PyPI long description, and the badge links fixed below could reach the project page only through a release.
Added
- GitHub Releases.
release.ymlcreates one for each tag once PyPI has the distributions: its notes are the version's section of this file, with links into the repository pinned to the tag, and it carries the same wheel and sdist. A tag whose version has no section here now fails before anything is published. Releases for 0.5.0 to 1.4.3 were created from this file. - A Read the Docs badge in
README.md. README.md's Python examples run in the documentation tests, as every page underdocs/already did. It is the page GitHub and PyPI show, and it was the only one whose examples nothing executed.
Fixed
README.mdon PyPI. The License and Typed badges were relative links, which resolve on GitHub but 404 on the PyPI project page, where the README is the long description; both are absolute now.README.md's "Write a sample" raisedNameErrorwhen pasted; it now defines the arrays it writes. "The reasoning behind each" pointed at the documentation's front page rather than the design rationale.
medh5 1.4.3
Released on 2026-09-27. This GitHub release was added on 2026-10-01.
A documentation release. No format change and no change to the package's behaviour: a review of every page against the code and the specification. It exists, like 1.1.1 before it, because README.md is the package's PyPI long description — the corrected install note and the new links below could only reach the project page through a release.
Removed
- The
design/records. The 1.0 design proposal and implementation plan were pre-implementation working documents, unmaintained by their own account. The reasoning that still holds — the problems 0.x could not express, the alternatives weighed, the costs accepted, the non-goals — is now a maintained page, Design rationale, and the benchmark results the proposal carried are on Runnable examples beside the scripts that produce them. Old links to the proposal redirect to the new page; the plan remains in the repository's history. Docstrings andmedh5 bench --helpno longer cite the plan.
Added
CONTRIBUTING.md: setup, the gates, how the specification and the tested documentation are kept in step with the code, and the release process.- Documentation for what existed but was undocumented:
from-nifti --fourth-axisand 4-D series; thegridandroiclassification scopes;identitytransforms;maskannotations andvalid_mask; the Python entry points of every converter;s.index,s.fresh_indices,s.label_set,Timeline.interval_days,commit()/abort(),remap_frame_uids,audit_splits,pack/unpack,grid_patches; the MONAI adapter'sspace=;PairedPatchDataset's change labels andsamples_per_pair. - A test holding the conformance page's breakdown of the corpus — valid, invalid, samples, collections, mutated — to the corpus. Only the total was checked, and the breakdown had drifted.
Fixed
- Specification, editorially. The heading for §5.4 (hierarchy and
closure) was missing although three clauses cite it; §6.3's kind registry pointed every geometric kind one subsection early; §4.2 and §8.5 cited §10.3 and §10.5 for §10.4 and §10.6; §2.4's table of/metamembers omittedquality. No requirement changed. - Claims the code contradicts.
dense([65535])is refused withE404, not answered with an all-zero plane; contours and meshes do not satisfy thedetprofile; the profile table now states §1.3's requirements; a classification'slabelsare keyed by class key;Cohorthasacquisition_protocol, notprotocol;transform_betweentraverses a link backwards only where an inverse can be evaluated;portableshuffles before gzip; the core install includesjsonschema. - Stale numbers. The conformance page's 39 valid cases, 111 samples and 71 mutated cases are 41, 113 and 82; the landing page and the training tutorial quoted foreground-sampling timings from before 1.4.2.
- The CLI reference now matches the parser:
fix --by,track --key,index builddefaults,dataset split --assigned-by,dataset stats --set-idand--stride,--reportand--jsonon the converters, and the default annotation ids.medh5 index build --helpnow says--occupancyis the occupancy map's block size, which is what it always was, not its side length. - Page titles that disagreed with the navigation, links labelled with pages that no longer exist, and a tutorial that duplicated the next one.
medh5 1.4.2
Released on 2026-09-26. This GitHub release was added on 2026-10-01.
The second patch release under the third audit's plan, and the last of it: the cohort, curation and registration tools, performance, and hygiene. No format change — 1.4.2 reads and writes the files 1.0 does, and the conformance corpus is still 117 cases. Most of what changes is a tool answering a question wrongly without saying so: an agreement score for a comparison that measured nothing, a split audit that missed one subject in two groups, a verified file whose attested object had gained an undigested dataset, a box read on another grid at the wrong size. After this release every one of the twelve "final 1.x" properties holds.
Behaviour changes
Read these before upgrading a pipeline. Each is a correction; each can change what existing code produces or accepts:
-
Agreement scores only what was measured (L-29). Object agreement now does what voxel agreement did: an unmatched object whose class the other annotation never examined is left out and listed under
skipped— identical work scored 0.667 when the second rater had not been asked to look for a class — while a matched pair always counts. When neither side has anything to compare,valueisNonerather than 0,mean_iouisNonewhen nothing matched, andto_record()refuses rather than writing a 0 into the file.compare_instancesrefuses twoindexannotations on different grids (E101), and boxes in different spaces or in different frames (E414): boxes on a 4 mm and a 1 mm grid were compared in raw index units.compare()picks the comparison the two kinds support and refuses an argument it cannot use;medh5 agreegoes through it, so twoboxesannotations no longer end in anAttributeError, and--metricor--thresholdon the wrong kind is an error rather than ignored. Comparisons across two visits' grids now raise. -
An agreement record can be stored in the file it describes (L-38).
to_record()keyedper_classby class name, and the object record put its mean IoU there asmean_iou; the schema admits only class ids, so every recordmedh5produced failed withE005when written. Records are keyed by class id now, and the object record carries noper_class. -
Duplicate ids are refused (L-30). A document that declares one agent or activity id twice is refused when read — the reader kept the last, and a no-op
amenddeleted the other from the file. The validator reports it through the parse (E005; a dedicated code is format 1.1). Two objects sharing aninstance_idinside oneinstancesannotation — which §7.4 forbids — are refused by the writer and reported by the validator asE404. Such third-party files no longer open or validate. -
Split audits find a subject stored under two
group_ids (L-33).medh5 splits,audit_splitsand the cohort checkC202grouped by the grouping key alone, so one subject in train and in test under two keys reportedleaks = 0, ok = True— the casemake_splitsrefuses asC204. Leaks are now found over subjects and keys together (anatomy_units, union–find), and aLeaklists the keys and subjects it joined. Audits can report leaks they used to miss. -
The frame graph (L-35, S-12). A transform and the stored inverse its
inverse_idnames are one route: mutually declared inverses — whatE505checks as "mutually consistent" — raisedE501in both directions, and the registration guide told users not to declare them. They now resolve both ways. A composite that contains itself isE501in the validator and when evaluated, where it validated clean and recursed untilRecursionError.transform_betweenraisesKeyErrorfor a key that is not a timepoint, a grid or a frame the file declares; a mistyped"TP1"returnedNone, the answer for "no registration exists". -
verifyfails a file whose attested object gained an undigested dataset (L-26).content_idcovers the digests present, soinstance_idsadded to a boxes annotation left the file verified and changedtracks(). In a file that declares acontent_id, an undigested dataset inside a grid, image, annotation or transform is now listed inVerifyResult.unattestedand makesokfalse (medh5 verifyandrecompressprint it asUNSIGNED;fixcounts it as needing digests). The writer digests every dataset, so none of its files is affected. -
Index coordinates belong to the annotation's own grid (L-32).
as_slices(grid=),to_index(grid=)andto_world(grid=)on anindex-space annotation read its coordinates as the other grid's: a box on a 4 mm grid asked for slices on a 1 mm grid of the same frame came back unchanged, covering a quarter of the anatomy. They now convert through world when the frames match and refuse (E414) otherwise; since §3.3 (1.4.1) a grid without aframe_uidrelates to nothing, which now applies toworld-space conversions onto another grid too. Boxes withslice_indexare read on their own grid only. -
RTSTRUCT rasterisation keeps an island inside a hole (Q-18). Every outer contour on a slice was unioned and every enclosed contour subtracted, so an island drawn inside a hole was erased. A contour's role now follows its nesting depth, and each slice is filled by the even-odd rule the provenance record already named. Rasterised masks gain those islands.
-
recompress --rechunkchunks the way the writer does (L-28) — from each grid'spatch_hintorchunk_hint,(1, *spatial)for stacked encodings — where it let h5py guess and spanned the stacked axis the spec keeps at 1, which the validator reported asW902. -
Sampling-index datasets are compressed (P-12) with the file's label codec, and
occupancyis chunked one class per chunk; it was uncompressed and contiguous. Files with an index get smaller. -
The CLI reports a missing name as an error, not a traceback (Q-12).
medh5 convert to-nifti FILE <unknown image>printed aKeyErrortraceback; any lookup error is nowmedh5: …and exit 1. -
SampleWriter.deidentification()with no fields is refused (Q-11). Anassertstood there, so underpython -Othe call wrote nothing and cleared an existing record.
Added
medh5.torch.FileGroupedSampler(P-11): shuffles files and yields each file's items together, so an epoch opens each file once per worker. At 100 small files and four items each, 304 opens became 100 and 0.77 ms per item became 0.30. Every dataset gainsfile_groups(), and the order follows the dataset'sset_epoch.medh5.torch.set_cache_sizeis exported and documented,HandleCache.resizecloses idle handles past the new size, andHandleCache.lease(path)holds a handle against eviction.compare(a, b, *, metric=, threshold=, classes=);compare_instances(..., classes=);InstanceAgreement.skipped;VoxelAgreement.class_ids;OBJECT_KINDS.VerifyResult.unattestedandDiagnosis.unattested;ATTESTED_GROUPS.Leak.groups,Leak.subjectsandmedh5.curation.splits.anatomy_units.SamplingIndex.has_class;medh5.storage.chunking.grid_chunks,fit_chunksandfield_chunks— the writer's chunk rule, shared withrecompress.medh5.transforms.resolve.stored_inverseandmedh5.transforms.composite.composite_cycle.medh5 benchgainsforeground_sample_many_ms, a foreground draw on a 63-class sample held to the same 1 ms target (T-11);medh5.bench.many_class_measurementandsynthetic_many_class_sample.
Fixed
- The indexed foreground draw is O(1) in the class count too (P-08).
SamplingIndexre-read its class table once per class it checked and twice per count it looked up: 190 HDF5 reads and 7.6 ms per draw at 63 classes. It reads them once per open, and a draw reads the one coordinate it picks: one read, 0.10 ms at 63 classes and 0.03 ms at eight (was 0.9). Draws are unchanged for a given seed. - Plane counts are bounded (P-09).
layersandbitmaskcounts read a whole plane before slabbing it — 256 MiB for auint16layer at 512³, 1 GiB for a bit-plane. The slab reader now descends an axis when one row is over its budget, for every bounded scan. - A transcode carries attributes it does not know (Q-13).
AnnotationHeader.readdropped them, so re-encoding an annotation lost anything a later minor version had added, whichamendcarries everywhere else (§16). - The handle cache is thread-safe (Q-16). Lookup and eviction are locked, and the datasets hold their handle for the length of an item, so a thread-based loader's eviction can no longer close a file another thread is reading.
medh5.monainames the channel axis (Q-17):original_channel_dimcomes from the grid, where"no_channel"on an RGB image madeEnsureChannelFirstadd a second channel axis.medh5 bench --jsonprints only JSON (Q-19). Its progress lines went to stdout ahead of the document; they go to stderr.recompressof a collection (L-39). A.medh5cshard was rewritten and then raised aKeyErrorwhile verifying the output, which it read as a single sample. Each member is verified.
Removed
medh5.geometry.grid.iter_spatial_slicesandKNOWN_COORD_SYSTEMS, which nothing used (Q-11); the duplicatetransforms.apply._refuse_outside.
Specification corrections (Appendix C, entry 21)
- §10.1 — a transform and the stored inverse its
inverse_idnames are one route between two frames, traversable both ways; a resolver must not count them as two (S-12).
Conformance, docs, tests and CI
- The
W909-instance-id-two-classescase puts its conflicting id in two annotations — the sample-scoped form Appendix C's §7.4 entry already requires — because inside one annotation a shared id is nowE404. Its expected codes are unchanged; the corpus is still 117 cases. - A test holds every place the corpus size is written down — README, site, conformance page, Appendix C, CI com...
medh5 1.4.1
Released on 2026-09-25. This GitHub release was added on 2026-10-01.
A patch release under the maintenance contract: defect and security fixes, and no format change — 1.4.1 reads and writes the files 1.0 does. A third full audit went after how the files are used: shipped to a collaborator after a scrub, fed to a training loop, recompressed by a curator, amended by a tool one version behind. It found twelve silent wrong answers and one security defect, all fixed here. Four fixes needed a little new API, because the defect cannot be fixed without it — loader item keys, classification rows, drop_identity, --pseudonymise-ids — and each is listed under Added. jsonschema becomes a core dependency. The corpus grows from 116 to 117 cases, and Appendix C now lists twenty corrected clauses.
Behaviour changes
Read these before upgrading a pipeline. Each is a correction; each can change what existing code produces or accepts:
-
Security: files that read outside themselves are refused by every tool (F-22). HDF5 lets a dataset keep its bytes in an arbitrary file on the reader's machine (external storage), and lets a file reach others through external links and virtual datasets. A crafted file whose extension dataset pointed at
~/.ssh/id_rsavalidated clean at--level strict, andrecompress --outcopied the key into the file a curator then shared.open,amend,recompress,repack,pack,verify,validateand the 0.x reader now walk the file first and refuse any such object by name, before reading a byte of it;validatereports E001. Copies no longer expand external data. The writer never produced these objects. -
scrubexamines the whole file and can exit 1 on files that used to pass (F-14, L-36). It kept a list of places to look, and each audit found string-bearing places the list had missed — this time provenanceinputs/outputs, the sample's own ids, timepoint text, per-object JSON, and every HDF5 attribute. It now walks every string in/meta, every object name, every attribute and every string dataset (JSON inside a string is decoded and examined as a mapping), skipping only fields named in the code with a reason, and the scan and--applyare one traversal, so "actionable" means what--applywill change. New rules: DICOM UIDs inside provenance references (dicom:<uid>) get the same pseudonym as the field they came from; filesystem paths keep only their file name, or nothing under--profile strict; ages over 90 and organization agents are reported and handled understrict; person names are recognised in any script (Müller^Hans). The DICOM importer records where the ids came from (identity.id_source),scrubreports asample_id/subject_idcopied fromPatientID, and a strict--applyexits non-zero while an id is still an identifier — re-mint it, or pass--pseudonymise-ids --salt .... A file named after such an id, or after a UID, is reported too. The attestation's profile string now says what was not examined. After a DICOM import and a strict apply with--pseudonymise-ids, no PatientID, UID or date bytes remain in the file. -
Loader items carry the ignore region, validity and coverage (F-16). Items gain
ignore[ann_id](the §7.7 region, wherever the encoding keeps it, and any padding),valid[image_id](the image'svalid_mask, never the padding) andmeta["annotated"][ann_id](one flag per label channel).label_format="labelmap"— andmedh5.monai.to_dict— now write 65535 at ignored voxels under every encoding, wherelayers,bitmask,instancesandprobmapgave 0, and at padding, which was 0 (background). The ignored voxels §7.7 exists to keep out of the loss went into it as negatives. -
Persistent workers draw new patches every epoch (F-17). The epoch was an
inton the dataset;DataLoader(persistent_workers=True)— the documented setup — kept the copy it started with, so every epoch repeated epoch 0's patches. It now lives in shared memory andset_epochreaches every worker, underforkandspawn.ds.epochstill reads and assigns as before. -
Two grids without a
frame_uidare not registered (F-18).PairedPatchDataset(align= "transform")compared the two frames with==, andNone == Nonelet every unregistered longitudinal sample from the NIfTI and nnU-Net importers through as aligned. It now requires a shared frame (§3.3) or a transform; such samples raise and say to usealign="none". -
A gantry-tilted DICOM stack is refused (F-19). It imported as an orthonormal grid, 17 mm off at the last of twenty slices at 20°. The refusal names the tilt and the worst slice's offset; a stack within half a pixel of its axis imports as before, with a
slice_alignmentdecision recorded. -
amendrefuses a future major version and keeps a later minor (F-20). It restamped a 2.0 file1.0, and a 1.1 file1.0while carrying its 1.1 objects.scrub --apply,fixandseg convertgo throughamend. -
Cohort class statistics come from voxel annotations, per sample (F-24). A sample-level classification arrived as "examined, 0 voxels", and
class_weights()floored it to 1 voxel — making a one-bit diagnosis the heaviest segmentation class. Classification, geometric andmaskannotations no longer contribute;present_in/examined_incount samples, not annotations; andclass_weights()leaves out classes with no voxels, with a warning naming them. Weights and prevalence change for cohorts with non-voxel annotations. -
Contradictory demographics split a PatientID (F-25). An anonymiser's constant
PatientIDmerged two patients into one subject with two visits. Studies whosePatientBirthDateorPatientSexdisagree now become one sample each, with a guess in the report naming both values. Such imports produce more files. Study-keyed files are named by their whole StudyInstanceUID, wherePath.stemused to chop its last component. -
commit()runs the validator (L-19–L-22). The writer and the validator each kept their own list of rules and drifted apart.commit()now runs the validator's structural and semantic error rules over the finished file and refuses to write one it would reject (warnings are not raised; measured at about 2 ms on a 192×256×256 sample). A displacement field off its grid's lattice is refused atadd_transform(E503); gridunitsoutsidemm,um,m,pxandtime_unitsoutsides,msare refused (E109); and aform="ref"label set is written, where the writer refused what the validator accepts. Files the validator would reject are no longer written. -
add_segmentationrefuses what it used to half-honour (L-23, L-24, L-37). Passing more than one ofmasks=,probabilities=,instances=, or anencoding=the argument cannot use, is E404 —masks=was silently dropped besideprobabilities=. An ignore region that overlaps a class is stored as the sibling mask under every encoding, so it reads back whole: underlabelmap56 of 64 voxels came back. Such files gain a sibling mask instead of an in-band region.instances=[]withannotated_classes=now writes the "examined, none found" annotation (§7.4) it used to refuse. -
A transcode from
instancesto a dense encoding needsdrop_identity=True(L-25). Every voxel survived and everyinstance_iddid not;tracks()then found nothing. With the flag (CLI--drop-identity) it proceeds and records atranscodeactivity saying identity was dropped. Callers that relied on the silent drop must pass the flag. -
amendkeeps aportablefile portable (L-27). New datasets were Blosc2, unreadable withouthdf5plugin.amend(codec=None)now means the source's codec family. -
unpackrefuses a member name that is a path (L-31). Names read from a shard were used as file names unchecked;..\..\evilwrote outside the output directory on Windows. -
Intensity moments honour
valid_mask(L-34), andtotal_voxelscounts each measured image's grid once. A CT padded outside its reconstruction circle reported a mean of −1180 where the tissue averages 40. Normalisation statistics change for images that declare a valid mask. -
A probability map keeps the voxels on its threshold (F-15). Stored as float16, a vote of 1 in 3 became 0.33325 and fell below a threshold of 1/3 — at write time. Readers now compare at the stored precision, and the writer widens to float32 where float16 would change an answer, so such maps take twice the bytes. Existing files read their boundary voxels as contained.
-
The validator no longer invents E411 and W904 on large labelmaps (F-21). It declined to look for an ignore voxel past 64M elements and then reported the
uint16the ignore id requires as an error —medh5 validatefailed the writer's own output at ordinary CT sizes. -
Every encoder refuses a class id outside 1–65534 with E303 (Q-14) before the
uint16cast, where numpy 2 raisedOverflowErrorand numpy 1.24 wrapped the id silently. -
jsonschemais a core dependency (L-21). Without it E005 went unchecked andcommit()wrote documents a machine with it rejects. Theschemaextra still installs, and adds nothing.
Added
Each is needed by a fix above, and none changes the format:
Sample.ignore_region(ann_id, roi=None)andSample.valid_region(image_id, roi=None)— the two reads the loaders use, public so a custom loader can make them.- Loader item keys
ignore,validandmeta["annotated"](F-16). add_classificationaccepts rows(class, value[, scope_id[, scheme, scheme_value]]), so one class can be asserted per lesion, slice or visit (F-23). The mapping form is unchanged.transcode(..., drop_identity=True),transcode_annotation(..., drop_identity=True)andmedh5 seg convert --drop-identity(L-25).scrub.apply(..., pseudonymise_ids=True)and `medh5 scrub --pseudonymise-i...
medh5 1.4.0
Released on 2026-09-04. This GitHub release was added on 2026-10-01.
The final 1.x release. The format version is unchanged: 1.4.0 reads and writes the files 1.0 does. What changes is what the writer does with what it is given. A second full audit went after the inputs rather than the outputs — an ignore region under an encoding chosen by measurement, a provenance id a caller picked, an export of a 2-D sample, a DICOM SEG written the way highdicom writes them by default — and found that the package produced good files while quietly dropping, overwriting or refusing several ordinary things asked of it. Those are the subject of this release. The corpus grows from 115 to 116 cases and Appendix C now lists sixteen corrected clauses.
After this, the package is in maintenance against format 1.0.
Behaviour changes
Read these before upgrading a pipeline. Each is a correction; each can change what existing code produces or accepts:
-
add_segmentation(ignore=...)now keeps the region under every encoding. It was forwarded to the encoder only forlabelmapandlayers; underbitmask,instancesandprobmapthe array was dropped without a word — no in-band value, noignore_maskattribute,has_ignore_regionreadingFalse. Withencoding="auto"the caller could not know which branch they were on, so the same call kept the region for one cohort and lost it for the next, and every ignored voxel became a verified negative for every annotated class with W904 unable to fire. The writer now stores the region where the chosen encoding can hold it: in band forlabelmapandlayers, and otherwise as a siblingmaskannotation named<ann_id>_ignore— same grid, sameprov— referenced by the header'signore_mask(§7.7). Files written withignore=under those three encodings gain that extra annotation. Passing bothignore=andignore_mask=is now E404, andencode_voxels, which returns a single payload and so cannot create the sibling, refuses rather than dropping.select_encodingtakesignore=and costs theuint16widening an in-band ignore forces, so the encoding is chosen with the region in view. -
Provenance ids are never silently overwritten.
Provenance.add_agentandadd_activityassigned into a dict, soperson("Alice", agent_id="s2")followed bysoftware("tool")— whose automatic id is alsos2— left one agent, the software, and every activity that named Alice named the tool. The file validated, because every reference resolved; it resolved to the wrong node. Both now refuse a duplicate id (replace=Truefor the one legitimate rewrite), and the writer's automatic ids skip taken ones. Code that relied on overwriting now raises. -
transcode_annotation(..., "mask")is refused (E404). Amaskhas no classes, so the conversion OR-ed every class into one volume and leftclass_idsandannotated_class_idsempty undertask="segmentation"— the coverage contract gone, in a file that still validated.TRANSCODABLEis now the whole truth; build a deliberate union withencode_maskandadd_mask. The CLI was already safe, sinceseg convert --torestricts its choices. -
scrub --applyacts on everything it flags, re-scans what it wrote, and can exit 1. It cleanedextraandacquisitiononly, then wrote a §11.4 record whose profile string reads "quasi-identifiers removed" and exited 0 — over files still carrying the patient name, institution and referring physician it had itself reported inidentity.extraandcohort. Three places were never scanned at all:activity.params(where a converter naturally storesOperatorsName),activity.tool, andquality.issues[].note. All are scanned and cleaned now, along with agent roles and qualifications; under--profile stricta person agent's name becomes a stable pseudonym and flagged free text is removed.applythen re-runs the rules over the file it wrote, recordsremainingandremaining_actionablein the deidentify activity's params, and the CLI exit code follows that re-scan.identity.sample_idandidentity.subject_idare reported and never rewritten — every manifest, split claim and cross-file join names the sample by them. -
Exported NIfTI files carry their rescale.
Image.read()returns stored values, and both exporters wrote them with noscl_slope/scl_inter: a CT imported from DICOM with intercept −1024 left every voxel 1024 HU too high, into_nnunetv2's only path and into_nifti(physical=False), with nothing indataset.jsonor the report to say so. Both now go through onewrite_nifti, which records the scale, so every conforming reader — nibabel, SimpleITK, and therefore nnU-Net — sees the physical values. The numbers a downstream tool reads from those files change, which is the correction. -
2-D samples can be exported.
convert_worldwas defined for 4×4 affines only, so both exporters stopped on a 2-D grid's 3×3 with "RAS↔LPS conversion is defined for 3-D affines" — and the import → export round trip the converters page promises was broken for every 2-D sample, although both importers accept one. It handles a 2-D affine now, andembed_planeputs it back in the 4×4 a NIfTI file carries (§3.6). -
A DICOM SEG that omits its empty frames imports.
omit_empty_frames=Trueis highdicom's default and the common form in the wild. The reader assembled a volume from the planes present — two slices, spaced by the distance between them — and the import compared that shape with the grid's and refused with E405 "drawn on a different reconstruction". Frames carry their ownImagePositionPatient, so they are now placed on the grid by position (§3.3), and shape agreement is a question about rows and columns. A frame that genuinely is not a slice of the target grid is still refused, now naming the frame.read_dicom_seg_framesandplace_framesare the new public entry points;read_dicom_segkeeps its shape and meaning. -
recompressverifies its output, and its exit code follows.content_id_preservedcompared the attribute the function had just copied with itself — true by construction, including on a file whose bytes were corrupted before the run, which reported "content_id yes", exited 0, and failedverify()immediately after. The re-encoded file is now read back and checked against the digests it carries. It takes roughly twice as long, and can now fail on a file that was already broken. -
commit(digests=False)keepscontent_id. It skipped the root address entirely, so an amend that used the flag for speed turned an addressed file into an unaddressed one. It now stamps the datasets that carry no digest — on an amend, the ones just written — and always computes the root.create(...).commit(digests=False)therefore stamps everything, since nothing has a digest yet. -
Timepoint.extra,QualityRecord.extraandActivity.extraare gone, andAgent.extrawith them. The schema closes all four objects, so anything placed there failed E005 atcommit(): an extension point that could not reach a file. An unknown key is now E005 where it is parsed, with the field named.Agentgains theorganizationfield the schema has always had, andTimepointgainsdescription. -
open_collectiontakes nomode.Collectionhas no mutating method, asmedh5.openstopped taking one in 1.3.0. -
dataset checkC202 groups by the §12.2 grouping key, asmedh5 splitsdoes, and names the subjects involved. A family or longitudinal group straddling two partitions was a LEAK in one tool and clean in the other, on the same files.
Fixed
info,tree,verify,timelineandtrackaccept a.medh5cshard with--key;infoandtreewithout one describe the shard. Six help strings promised collection support the commands did not have.Sample.reference_gridraises E111 on a file with no grids, where it raised a bareIndexError.to_nnunetv2lists inreport.outputsonly the files it wrote; it named alabelsTrpath whether or not the sample had the annotation.recompressno longer copies the root attributes twice.- The annotation reference gave
closureas"explicit" | "complete"; the values areexplicitandimplicit.seg convertis documented with the five targets it accepts.
Performance
layersandbitmaskcache their class tables per open annotation, asinstanceshas since 1.3.0.dense()asked the table once per class: 63 HDF5 reads of the same table for a 63-class annotation packed into four layers.storage.index._occupancyis vectorised — pad, reshape,anyover the block axes — where it was a Python loop over every coarse block: about 80 ms per class at 256³ and 0.6 s at 512³, sobuild_indexon a 200-class annotation spent minutes on a map measured in kilobytes.voxel_counts()answers every class in one pass where the encoding allows it: a slabbincountforlabelmapandlayers, a per-plane population count forbitmask. It decoded each class separately, andbuild_indexand the unindexed path ofdataset statsboth go through it.
Specification corrections (Appendix C, entry 16)
- §7.7 names
instancesbesidebitmaskandprobmapas an encoding whose ignore region must live in a separatemaskannotation. It has no in-band value either, so the clause left a conforming writer with nowhere to put one.
Conformance and tests
- The corpus grows to 116 cases, with
seg-bitmask-ignore-mask: §7.7's separate-mask form as the writer now emits it, partial coverage and W904 silent. tests/v1/test_release_1_4_0.pycarries every reproduction above, each named for the finding it came from, and each failing on 1.3.0.tests/v1/test_docs_python.pyexecutes every Python example in the documentation against a real sample.test_docs_examples.pychecked the prose and the CLI flags but nev...