Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
archive/zip: cannot parse file header with compressed size or local file header offset of 0xffffffff #31692
archive/zip misinterprets (I believe) APPNOTE.TXT 4.5.3, such that it wrongly requires a Zip64 Extended Information extra field to be present whenever the compressed size or local file header offset of a central directory header is exactly 0xffffffff. #14185 fixed the problem for the uncompressed size as a special case, but really there is nothing special about the uncompressed size and all three fields should be treated equally.
APPNOTE.TXT 4.5.3 says:
archive/zip interprets the statement as:
But that logic is backwards—it's an "only if", not an "if". I think the interpretation should rather be
In other words, 0xffffffff, by itself, is not a magic value that indicates special handling is required. It is the presence of a Zip64 Extended Information extra field that indicates special handling, and only then does the value 0xffffffff become significant. 0xffffffff is a perfectly valid field value to have in a non-Zip64 file.
I'm attaching a zip file that demonstrates the problem, ffffffff.zip.gz.gz. (It is gzipped twice to reduce the size of the attachment, but the gzip layers have nothing to do with the issue and you should remove them before testing.) The zip file was produced by Info-ZIP Zip 3.0 and contains 2 files, with a maximum compressed/uncompressed size of 0xffffffde and a maximum local file header offset of 0xffffffff. Zip has decided to write a non-Zip64 zip file, as none of the values exceeds 0xffffffff. Info-ZIP UnZip 6.00 can parse the file, but archive/zip cannot. The sample file was created as follows:
archive/zip doesn't have a problem if the local file header appears one byte earlier or later—the easiest way to test that is to use a 2- or 4-byte filename instead of "pad" in the recipe above. In the former case it's because the value is 0xfffffffe and in the latter case it's because the value is 0xffffffff but Zip64 information is present.
For corroboration, see the function
Fixing this issue will allow removing the special case introduced in #14185 because it will be handled by the general case: a value of 0xffffffff means what it says, in the absence of a Zip64 extra field.
This issue is only a problem when reading a zip file, not when writing. archive/zip currently writes Zip64 information whenever a field is exactly 0xffffffff—that's probably a good idea for interoperability, even if it's not required. Compare Zip 3.0's strict inequality (function
with archive/zip's non-strict inequality:
What version of Go are you using (
The full comment on the relevant code says:
I think this code is probably still best as written: no real zip encoder is going to write out 2³²-1 compressed bytes that uncompress to 2³²-1 or fewer bytes (and if it uncompressed to more it would need a zip64 header). The far more likely possibility is that the input is somehow corrupted or malformed (ErrFormat). That justifies the needCSize check.
The needHeaderOffset check is maybe slightly more debatable, but even so it still seems incredibly implausible and far more likely to be an invalid (or malicious) file than an innocently-created actual zip file. I think we should probably leave the code as is.
I think you're right about that. Compressing random data could result in a compressed size greater than the uncompressed size, but most encoders will switch to method 0 (Store) whenever that happens, so it's unlikely to be the case that
I don't think this one is so implausible. Any time the input consists of multiple files that total more than 4 GB, Info-ZIP Zip will store all the local file header offsets that are ≤ 0xffffffff without Zip64, and those that are > 0xffffffff with Zip64. It will be up to chance whether a local file header happens to land exactly on 0xffffffff and result in a zip file that archive/zip cannot parse. If the input consists of small files of around 50 bytes, then there's around a 1% chance of that happening.
I don't mean to overstate the importance, and I won't be upset if the issue gets closed with no changes. It's only a very small minority of zip files that will ever have an offset of exactly 0xffffffff. (Then again, you could say the same for 0xfffffffe.)