Skip to content

v0.57.0: raw tensor storages in PyTorch zip files are no longer read as pickles

Choose a tag to compare

@munimv2 munimv2 released this 10 Nov 02:22
· 37 commits to main since this release

model-file-scan 0.13.2, pickle_model_scan 0.6.0. Made with AI agents working in the name of MV2 of Munim, Inc.

  • pickle_model_scan 0.6.0: in a zip-format PyTorch file, torch.load unpickles only <prefix>/data.pkl and copies each tensor storage <prefix>/data/<key> as raw bytes. Up to 0.5.0 the scanner walked any zip member whose first byte is 0x80 as a pickle, so a storage that happened to start with 0x80 (for example BatchNorm's num_batches_tracked after 128 batches) got a false PKL000 LOW saying the stream could not be fully parsed. 0.6.0 skips only that first-byte sniff, and only for <prefix>/data/<key> members of a zip that holds <prefix>/data.pkl. Every data.pkl, every member with a pickle extension and every zip without a data.pkl is read exactly as before. No rule or severity changed.
  • model-file-scan 0.13.2: version bump only.

Pre-registered check zip-data-v1 (REALFILE_CHECK_ZIP_DATA_2026-11.md), graded against what torch.load reads on files torch wrote: all 11 release rows held. On 229 files the only difference between 0.5.0 and 0.6.0 is the removed storage-member findings. 8 of the 21 most downloaded zip-format pytorch_model.bin files we read (38%) got the false note from 0.5.0 and none from 0.6.0. Two graded, non-gating predictions failed: only 21 published files fitted the download budget (predicted at least 30), and 25 of 3,658 published storages (0.68%) start with 0x80 (predicted 0.2-0.6%). Full suite on the patched tree: 1,431 tests, 0 failures.

Still open, each queued for its own check: an honest TorchScript file gets PKL003 MEDIUM for its own __torch__ class names, and a torch.export .pt2 can get a PKL000 LOW on non-pickle members.

A clean scan is not a guarantee of safety: pickle-based formats can run code when loaded, so only load files from sources you trust. Apache-2.0.