Fix multi-grid standalone slide embedding - #7
Merged
Conversation
CraigMyles
marked this pull request as ready for review
July 11, 2026 22:25
This was referenced Jul 11, 2026
CraigMyles
added a commit
that referenced
this pull request
Jul 13, 2026
## Summary - pass each registry `weights_revision` to Hugging Face loaders whose API accepts a revision - for CONCH, KRONOS, OpenCLIP, and MADELEINE, download the pinned revision first and give the upstream factory a local path - verify the recorded SHA-256 before those local checkpoints reach an upstream deserializer - pin BiomedCLIP's nested PubMedBERT configuration to an immutable revision and make both model construction and tokenizer configuration resolve locally - pin every mutable `git+https` installation example to the audited upstream commit - require OpenCLIP 3.x for its supported `local-dir:` loading contract ## Why The registry already records immutable model revisions, but several loaders and install examples did not enforce them. A store could therefore record one revision while an upstream factory or a copied install command resolved the repository's current default branch. BiomedCLIP also contains a transitive dependency: its pinned top-level `open_clip_config.json` names a separate mutable PubMedBERT repository. OpenCLIP builds that text tower even though raw2features exposes only `encode_image`, so the nested configuration must be pinned as well. ## Implementation - `ClipHFEmbedder` forwards `revision=` to `AutoModel.from_pretrained`. - CONCH and KRONOS download their exact checkpoint files at the registry revision, verify their registry digest, and pass only the verified local path upstream. - QuiltNet and BiomedCLIP use a raw2features-owned, model-and-revision-specific cache containing only `open_clip_config.json` and `open_clip_pytorch_model.bin`; the exact `.bin` is verified before OpenCLIP loads `local-dir:<path>`. - BiomedCLIP downloads `config.json`, `tokenizer_config.json`, and `vocab.txt` from `microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract` at `d673b8835373c6fa116d6d8006b33d48734e305d`, then atomically rewrites the app-owned OpenCLIP config so both nested names are absolute local paths. The shared Hugging Face cache is never modified. - MADELEINE preserves `RAW2FEATURES_MADELEINE_DIR` and the existing default cache behavior, but downloads `model_config.json` and `model.pt` at the recorded revision, verifies `model.pt`, and calls the upstream local-path factory on the requested device. - VCS install instructions now pin CONCH, KRONOS, MUSK, MADELEINE, and Prov-GigaPath; the already-pinned SEAL example now also pins its CONCH backbone example. ## Compatibility There is no CLI or embeddings-store schema change. Model outputs, preprocessing, and recorded provenance fields are unchanged; the existing pins are now enforced during download and installation. OpenCLIP users need `open_clip_torch>=3.0`, which provides the local-directory loader used here. ## Validation - `333 passed, 50 deselected` in the full non-slow suite after rebasing onto merged PR #7, on Linux/Python 3.12 with the optional dependencies installed - `22 passed` in the focused hub and loader-pinning tests - `ruff check .` - `git diff --check` - independent review specifically checked the upstream loader contracts, pinned VCS revisions, transitive BiomedCLIP dependency, local-cache isolation, and deserialize-before-verification boundary; its blocking findings were fixed before the final test run - no gated real-weight/GPU run was performed for this branch; loader contracts are covered with isolated tests and upstream-source review Closes #5 Part of #6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
--gridsupport and deterministic grid inference for standalone slideencoding
level0_patchto position-aware encoders such as TITAN andGigaPath
inline and standalone paths
intentional
--forcereplacementmodel loading, or encoding
Why
The standalone command still assumed a single grid and called position-aware encoders
without
patch_size_lv0. Its duplicated implementation had also drifted from the inlinepath, while inline reruns without receipts could fail when
slide/<model>alreadyexisted.
Impact
This adds one backward-compatible CLI option,
--grid. Single-grid stores continue towork without it. Multi-grid stores infer a unique grid from the requested patch model
where possible and otherwise return an actionable error. There is no store schema
change.
Existing slide outputs are skipped only when their array shape, dtype, role, patch-model
provenance, dimension, mirrored header metadata, and values are valid. A stale or
malformed output is recomputed. Standalone encoding refreshes consolidated metadata once
after the encoder loop; inline encoding does so once when the sink closes.
Root cause
Inline and standalone slide encoding maintained separate feature-selection,
spatial-argument, provenance, and write logic. The standalone copy predated multi-grid
stores and the
patch_size_lv0encoder contract.Verification
241 passed, 48 skipped, 14 deselectedwithpytest -q -m "not slow"ruff check .git diff --checkslide-embed --helpuv buildconch_v1_5 @512grid recordedlevel0_patch=1024; standaloneslide-embed -s titanpassed 1024 to the encoder andproduced a finite
(1, 768)float32slide/titanvector--patch-model uniwas rejected in the real-weight workflowCloses #2
Part of #6