Check sha256 of weights

[adamjstewart]

@NicolasHug @nps1ngh @pmeier would this be sufficient to close #7210? This should be fully backwards compatible in case someone wants to use weights without a sha256 hash suffix. We'll rename all of our weights in TorchGeo to add the suffix.

[pmeier]

I'll leave it up to @NicolasHug to review this with respect to torch.hub, since he is the specialist here. From my naive understanding, this should be an innocent addition that could improve security.

[adamjstewart]

Ping @NicolasHug

[NicolasHug]

After the release please @adamjstewart :) #7210 (comment)

[adamjstewart]

Would love to get this into the release but if it's past the feature freeze it can wait.

[pmeier]

@adamjstewart Branch cut was last Friday, so it won't make it into the next release.

[adamjstewart]

@NicolasHug congrats on the new release! I know there's always fallout from new releases, but once that settles down, a gentle reminder to review this PR. I rebased so we can test it against the latest version.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/vision/7219

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• Degradation on most runner types due to networking outage

❌ 28 New Failures, 2 Unrelated Failures

As of commit c9a1b8a:

NEW FAILURES - The following jobs have failed:

• build / linux-job (gh)
• c-source / linux-job (gh)
• onnx / linux-job (gh)
• python-source-and-configs / linux-job (gh)
• python-types / linux-job (gh)
• pytorch/vision / manywheel-py3_8-cpu (gh)
• pytorch/vision / manywheel-py3_8-cuda11_7 (gh)
• pytorch/vision / manywheel-py3_8-cuda12_1 (gh)
• unittests (3.10, linux.12xlarge, cpu) / linux-job (gh)
• unittests (3.10, macos-12) / macos-job (gh)
• unittests (3.10, windows.4xlarge, cpu) / windows-job (gh)
• unittests (3.11, linux.12xlarge, cpu) / linux-job (gh)
• unittests (3.11, macos-12) / macos-job (gh)
• unittests (3.11, windows.4xlarge, cpu) / windows-job (gh)
• unittests (3.8, linux.12xlarge, cpu) / linux-job (gh)
• unittests (3.8, linux.g5.4xlarge.nvidia.gpu, cuda, 11.7) / linux-job (gh)
• unittests (3.8, macos-12) / macos-job (gh)
• unittests (3.8, macos-m1-12) / macos-job (gh)
• unittests (3.8, windows.4xlarge, cpu) / windows-job (gh)
• unittests (3.9, linux.12xlarge, cpu) / linux-job (gh)
• unittests (3.9, macos-12) / macos-job (gh)
• unittests (3.9, windows.4xlarge, cpu) / windows-job (gh)
• unittests-extended / linux-job (gh)
• unittests-prototype (3.10, linux.12xlarge, cpu) / linux-job (gh)
• unittests-prototype (3.11, linux.12xlarge, cpu) / linux-job (gh)
• unittests-prototype (3.8, linux.12xlarge, cpu) / linux-job (gh)
• unittests-prototype (3.8, linux.g5.4xlarge.nvidia.gpu, cuda, 11.7) / linux-job (gh)
• unittests-prototype (3.9, linux.12xlarge, cpu) / linux-job (gh)

BROKEN TRUNK - The following jobs failed but were present on the merge base 8811c91:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pytorch/vision / manywheel-py3_8-rocm5_3 (gh)
• pytorch/vision / manywheel-py3_8-rocm5_4_2 (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[adamjstewart]

Ping @NicolasHug

[adamjstewart]

Ping ping @NicolasHug

[NicolasHug]

Thanks for the ping and for your patience @adamjstewart .

PR looks good, I agree it should be enough to enable sha checks on torchgeo's end.

Before mergning, do you mind updating the rest of the weights.get_state_dict(...) calls to use check_hash everywhere?

The remaining ones should be these:

(pt) ➜  vision git:(weights/hash) ✗ git grep "get_state_dict(pro.*ss)"
torchvision/models/detection/faster_rcnn.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/detection/faster_rcnn.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/detection/faster_rcnn.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/detection/fcos.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/detection/keypoint_rcnn.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/detection/mask_rcnn.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/detection/mask_rcnn.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/detection/retinanet.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/detection/retinanet.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/detection/ssd.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/detection/ssdlite.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/optical_flow/raft.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/quantization/googlenet.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/quantization/inception.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/quantization/mobilenetv2.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/quantization/mobilenetv3.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/quantization/resnet.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/quantization/shufflenetv2.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/segmentation/deeplabv3.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/segmentation/deeplabv3.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/segmentation/deeplabv3.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/segmentation/fcn.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/segmentation/fcn.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/segmentation/lraspp.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/video/mvit.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/video/resnet.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/video/s3d.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/models/video/swin_transformer.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/prototype/models/depth/stereo/crestereo.py:        model.load_state_dict(weights.get_state_dict(progress=progress))
torchvision/prototype/models/depth/stereo/raft_stereo.py:        model.load_state_dict(weights.get_state_dict(progress=progress))

If that makes it easier for you, feel free to just revert the changes you made to the calls to get_state_dict() in this PR and we can address them all in a later PR.

[adamjstewart]

@NicolasHug I believe I got all the remaining ones, let me know if I missed anything.

[NicolasHug]

Thanks @adamjstewart , the PR LGTM. Our CI is a bit toasted right now so I'll try to merge it some time later. Nothing more to do on your side though :)

Thanks!