Enforce recorded model revision pins - #8
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
weights_revisionto Hugging Face loaders whose API accepts arevision
give the upstream factory a local path
deserializer
both model construction and tokenizer configuration resolve locally
git+httpsinstallation example to the audited upstream commitlocal-dir:loading contractWhy
The registry already records immutable model revisions, but several loaders and install
examples did not enforce them. A store could therefore record one revision while an
upstream factory or a copied install command resolved the repository's current default
branch.
BiomedCLIP also contains a transitive dependency: its pinned top-level
open_clip_config.jsonnames a separate mutable PubMedBERT repository. OpenCLIP buildsthat text tower even though raw2features exposes only
encode_image, so the nestedconfiguration must be pinned as well.
Implementation
ClipHFEmbedderforwardsrevision=toAutoModel.from_pretrained.verify their registry digest, and pass only the verified local path upstream.
containing only
open_clip_config.jsonandopen_clip_pytorch_model.bin; the exact.binis verified before OpenCLIP loadslocal-dir:<path>.config.json,tokenizer_config.json, andvocab.txtfrommicrosoft/BiomedNLP-BiomedBERT-base-uncased-abstractatd673b8835373c6fa116d6d8006b33d48734e305d, then atomically rewrites the app-ownedOpenCLIP config so both nested names are absolute local paths. The shared Hugging Face
cache is never modified.
RAW2FEATURES_MADELEINE_DIRand the existing default cachebehavior, but downloads
model_config.jsonandmodel.ptat the recorded revision,verifies
model.pt, and calls the upstream local-path factory on the requesteddevice.
the already-pinned SEAL example now also pins its CONCH backbone example.
Compatibility
There is no CLI or embeddings-store schema change. Model outputs, preprocessing, and
recorded provenance fields are unchanged; the existing pins are now enforced during
download and installation. OpenCLIP users need
open_clip_torch>=3.0, which providesthe local-directory loader used here.
Validation
333 passed, 50 deselectedin the full non-slow suite after rebasing onto mergedPR Fix multi-grid standalone slide embedding #7, on Linux/Python 3.12 with the optional dependencies installed
22 passedin the focused hub and loader-pinning testsruff check .git diff --checkrevisions, transitive BiomedCLIP dependency, local-cache isolation, and
deserialize-before-verification boundary; its blocking findings were fixed before
the final test run
covered with isolated tests and upstream-source review
Closes #5
Part of #6