Skip to content

5.9.0

Choose a tag to compare

@clemsgrs clemsgrs released this 06 Sep 13:49
· 56 commits to main since this release

This release makes per-slide extraction results available to callers as they land rather than only at the end of a run, clears a crash in distributed HuggingFace authentication, and aligns preprocessing with hs2p 4.4.3.

Added

  • Per-slide completion callback. Pipeline.run_with_coordinates and Model.embed_tiles accept an optional on_slide_persisted callable, invoked in the calling process once per persisted (sample_id, annotation) unit with that unit's TileEmbeddingArtifact (or HierarchicalEmbeddingArtifact), after the artifact is complete on disk. With num_gpus > 1 it fires as each rank reports a finished slide, so a caller can act on one slide while the others are still running — no more splitting extraction into repeated calls that relaunch torchrun and reload the encoder on every rank. Zero-tile slides, resume-skipped slides, and failed slides do not fire it; an exception raised inside it propagates out of the entry point. Omitting the argument leaves every return value unchanged. (#311)

Fixed

  • Distributed extraction no longer dies during HuggingFace login. load_model called huggingface_hub.login() on every model load. Under distributed extraction every rank re-ran it per chunk, and login() truncates and re-reads the shared stored_tokens file without a lock, so concurrent ranks raced and failed with ValueError: Token ... not found. load_model now exports an explicit token= as HF_TOKEN for the process and never touches disk. Gated weights still authenticate — but note that nothing logs in implicitly any more, so set HF_TOKEN (or pass token=) when pulling gated repos. (#308)

Changed

  • Requires hs2p >= 4.4.3. The hs2p public API is unchanged from 4.4.2; the release is preprocessing-internal, lowering tiling and mask-decoding memory use. Existing coordinate and embedding artifacts stay valid. (#312)
  • Codebase cleanup. 208 net lines removed across 20 files — unused inference imports, unreachable fallbacks, redundant CUDA detection and window helpers, repeated process-list reads, and tests keyed to filenames or line counts. Inference, artifact, and configuration behavior is unchanged. (#309)