Repository navigation
v0.56.0: legacy-format PyTorch files and .nemo checkpoints are read to the last pickle
model-file-scan 0.13.1, pickle_model_scan 0.5.0, nemo_container_scan 0.3.0. Made with AI agents working in the name of MV2 of Munim, Inc.
- pickle_model_scan 0.5.0 walks every pickle the loader reads, not just the first.
torch.save(..., _use_new_zipfile_serialization=False)(the legacy format, the default before PyTorch 1.6, still read bytorch.load) writes five pickles back to back and then raw tensor bytes. Up to 0.4.0 the scanner read only the 15-byte magic-number pickle and reported such a file clean without seeing any import. Files written with severalpickle.dumpcalls are now walked one stream after another. New keysstreamsandtorch_legacyinpickle_walk(). No new rule and no severity change. - nemo_container_scan 0.3.0 does the same for a legacy-format
model_weights.ckptinside a.nemotar. 0.2.0 reported nothing at all for one, not even the NEMO010 note a zip-format checkpoint gets. It now reports NEMO010 with "legacy torch.save, 5 of 5 pickles". - model-file-scan 0.13.1: version bump only. Routing and every other scanner are unchanged.
Pre-registered checks, graded against torch.load / pickle.load on honest files:
- legacy-torch-v1, 19 of 19 held (REALFILE_CHECK_LEGACY_TORCH_2026-11.md). 0.5.0 ends its walk exactly where the loader's last pickle ends and resolves every import the loader asked for on all 8 files with ground truth. Among the 500
pytorch_model.binheaders read from the 1,000 most downloadedpytorchmodels on the Hugging Face Hub, 79 (15.8%) are legacy format, among them gpt2, bert-base-uncased and t5-small. On four small ones 0.4.0 read 15 bytes and 0.5.0 read the whole pickle. - nemo-legacy-v1, all 12 release rows held (REALFILE_CHECK_NEMO_LEGACY_2026-11.md). All 280 weights members in 55 published
.nemofiles are zip format, none legacy, so this closes a silent case for private or older checkpoints.
Each fix's full suite ran on its own tree (1,396 and 1,397 tests, 0 failures). Before commit, the changed modules' tests ran together on this release's files.
A clean scan is not a guarantee of safety: pickle-based formats can run code when loaded, so only load files from sources you trust. Apache-2.0.