Skip to content

Commit

Permalink
building: util: Read data for cache digest in chunks.
Browse files Browse the repository at this point in the history
This fixes MemoryError when hashing big files:
  File "c:\python27\lib\site-packages\PyInstaller\building\utils.py", line 333, in cacheDigest
    data = open(fnm, "rb").read()
MemoryError
  • Loading branch information
shawnluo-528 authored and htgoebel committed Feb 26, 2018
1 parent 477e0e6 commit 1934b77
Showing 1 changed file with 4 additions and 2 deletions.
6 changes: 4 additions & 2 deletions PyInstaller/building/utils.py
Expand Up @@ -338,8 +338,10 @@ def checkCache(fnm, strip=False, upx=False, dist_nm=None):


def cacheDigest(fnm, redirects):
data = open(fnm, "rb").read()
hasher = hashlib.md5(data)
hasher = hashlib.md5()
with open(fnm, "rb") as f:
for chunk in iter(lambda: f.read(16 * 1024), b""):
hasher.update(chunk)
if redirects:
redirects = str(redirects)
if is_py3:
Expand Down

0 comments on commit 1934b77

Please sign in to comment.