zipfile.py - pack filesize as unsigned allows files > 2 gig

[laurelboa]

BPO	679953
Nosy	@brettcannon

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2004-07-10.19:20:14.000>
created_at = <Date 2003-02-04.01:54:29.000>
labels = ['library']
title = 'zipfile.py - pack filesize as unsigned allows files > 2 gig'
updated_at = <Date 2004-07-10.19:20:14.000>
user = 'https://bugs.python.org/laurelboa'

bugs.python.org fields:

activity = <Date 2004-07-10.19:20:14.000>
actor = 'brett.cannon'
assignee = 'none'
closed = True
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2003-02-04.01:54:29.000>
creator = 'laurelboa'
dependencies = []
files = []
hgrepos = []
issue_num = 679953
keywords = []
message_count = 2.0
messages = ['14417', '14418']
nosy_count = 2.0
nosy_names = ['brett.cannon', 'laurelboa']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue679953'
versions = ['Python 2.2']

[laurelboa]

Python 2.2.2
Windows XP (all serice packs installed)
Windows 2000 (all service packs installed)

The filesize and compressed file size numbers in the zip
header need to "struct.packed" as unsigned ints, not
signed ints. This allows zipfile.py to compress files
greater than 2 gigabytes in size. Currently, an attempt
to compress such a large file gives you this error:

Traceback (most recent call last):
  File "<interactive input>", line 1, in ?
  File "C:\Python22\lib\zipfile.py", line 426, in write
    zinfo.file_size))
OverflowError: long int too large to convert to int

where the line in question is:
self.fp.write(struct.pack("<lll", zinfo.CRC,
zinfo.compress_size,
zinfo.file_size))

I believe that the four changes below are all that is
needed. This is from version 2.2.2, but zipfile.py in 2.3a1
still had the file size packed/unpacked as a signed
integer.

I have not tested whether the ziplib routines can seek
past the 2 gig boundary in order to extract a file whose
beginning is past the 2 gig boundary. My application
requires compressing very large files one at a time and
zipfile.py lets me use either WinZip or the built-in
Windows "unzip" function for extraction.
These changes allow that use.

-------------- Change Line #28

# Here are some struct module formats for reading
headers
structEndArchive = "<4s4H2lH" # 9 items, end of
archive, 22 bytes
stringEndArchive = "PK\005\006" # magic number for
end of archive record
structCentralDir = "<4s4B4H3l5H2l"# 19 items, central
directory, 46 bytes

to

structCentralDir = "<4s4B4HlLL5H2L"# 19 items, central
directory, 46 bytes

--------------- change line #306

    def printdir(self):
        """Print a table of contents for the zip file."""
        print "%-46s %19s %12s" % ("File 
Name", "Modified    ", "Size")
        for zinfo in self.filelist:
            date = "%d-%02d-%02d %02d:%02d:%02d" % 
zinfo.date_time
            print "%-46s %s %12d" % (zinfo.filename, date, 
zinfo.file_size)
to 
            print "%-46s %s %12u" % (zinfo.filename, date, 
zinfo.file_size)

---------------- change line #425

        # Seek backwards and write CRC and file sizes
        position = self.fp.tell()       # Preserve current 
position in file
        self.fp.seek(zinfo.header_offset + 14, 0)
        self.fp.write(struct.pack("<lll", zinfo.CRC, 
zinfo.compress_size,
              zinfo.file_size))
to
        self.fp.write(struct.pack("<lLL", zinfo.CRC, 
zinfo.compress_size,
              zinfo.file_size))

---------------- change line #450
if zinfo.flag_bits & 0x08:
# Write CRC and file sizes after the file data
self.fp.write(struct.pack("<lll", zinfo.CRC,
zinfo.compress_size,
zinfo.file_size))

to
self.fp.write(struct.pack("<lLL", zinfo.CRC,
zinfo.compress_size,
zinfo.file_size))

[brettcannon]

Logged In: YES
user_id=357491

Fixed in rev. 1.33 for Python 2.4.a2 and rev. 1.31.8.1 for Python 2.3.5 .
Thanks, Jimmy. Just added to your fixes but they did help.