* S3/Utils.py (hash_file_md5): Hash files in 32kB chunks

  instead of reading it all up to a memory first to avoid
  OOM on large files.



git-svn-id: https://s3tools.svn.sourceforge.net/svnroot/s3tools/s3cmd/trunk@208 830e0280-6d2a-0410-9c65-932aecc39d9d

ChangeLog

@@ -1,3 +1,9 @@
+2008-07-29  Michal Ludvig  <michal@logix.cz>
+
+	* S3/Utils.py (hash_file_md5): Hash files in 32kB chunks
+	  instead of reading it all up to a memory first to avoid
+	  OOM on large files.
+
 2008-07-07  Michal Ludvig  <michal@logix.cz>
 
 	* s3cmd.1: couple of syntax fixes from Mikhail Gusarov

S3/Utils.py

@@ -139,7 +139,12 @@ def mktmpfile(prefix = "/tmp/tmpfile-", randchars = 20):
 def hash_file_md5(filename):
 	h = md5.new()
 	f = open(filename, "rb")
-	h.update(f.read())
+	while True:
+		# Hash 32kB chunks
+		data = f.read(32*1024)
+		if not data:
+			break
+		h.update(data)
 	f.close()
 	return h.hexdigest()