Skip to content

Commit

Permalink
Replace hashlib.sha256 with hashlib.file_digest so we don't need to l…
Browse files Browse the repository at this point in the history
…oad entire files into ram before hashing them. (oobabooga#4383)
  • Loading branch information
LightningDragon committed Oct 25, 2023
1 parent 72f6fc6 commit 0ced78f
Showing 1 changed file with 1 addition and 2 deletions.
3 changes: 1 addition & 2 deletions download-model.py
Original file line number Diff line number Diff line change
Expand Up @@ -236,8 +236,7 @@ def check_model_files(self, model, branch, links, sha256, output_folder):
continue

with open(output_folder / sha256[i][0], "rb") as f:
bytes = f.read()
file_hash = hashlib.sha256(bytes).hexdigest()
file_hash = hashlib.file_digest(f, "sha256").hexdigest()
if file_hash != sha256[i][1]:
print(f'Checksum failed: {sha256[i][0]} {sha256[i][1]}')
validated = False
Expand Down

0 comments on commit 0ced78f

Please sign in to comment.