-
Notifications
You must be signed in to change notification settings - Fork 15
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Detect and reject a zip bomb using overlapped entries.
This detects an invalid zip file that has at least one entry that overlaps with another entry or with the central directory to the end of the file. A Fifield zip bomb uses overlapped local entries to vastly increase the potential inflation ratio. Such an invalid zip file is rejected. See https://www.bamsoftware.com/hacks/zipbomb/ for David Fifield's analysis, construction, and examples of such zip bombs. The detection maintains a list of covered spans of the zip files so far, where the central directory to the end of the file and any bytes preceding the first entry at zip file offset zero are considered covered initially. Then as each entry is decompressed or tested, it is considered covered. When a new entry is about to be processed, its initial offset is checked to see if it is contained by a covered span. If so, the zip file is rejected as invalid. This commit depends on a preceding commit: "Fix bug in undefer_input() that misplaced the input state."
- Loading branch information
Showing
5 changed files
with
205 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
47b3cea
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please note the comment in the commit, that this depends on another commit. You cannot apply this commit in isolation. Also there is a subsequent commit that generalizes the bomb detection to zip-like containers that do not follow the zip standard, putting the central directory at the beginning of the container.
47b3cea
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hello,
I am going through the patch, and I am a bit confused by the type of the
Globals.cover
. Is there any reason for it being the double pointer? Shouldn't it be justvoid *cover;
, or am I missing something?Thanks,
Jakub
47b3cea
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, nevermind, It's a double pointer so you are able to free the span list. Sorry for spam:-)
47b3cea
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hello
After I merged
41beb47
47b3cea
6d35183
the decompression of my local non-bomb file failed, and the prompt as follows:
[test@localhost /home]# unzip MyTxxx.zip
Archive: MyTxxx.zip
creating: scripts/
creating: scripts/ixx/
inflating: scripts/ixx/change_xxx.sh
error: invalid zip file with overlapped components (possible zip bomb)
But when use the old version to decompress, the decompression was normal.
Will these three patches break the decompression behavior of the original compressed file? Or a new constraint change?
Thanks.
47b3cea
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you make the file in question available for me to look at?
47b3cea
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks for your reply。
the zip file is compressed by a java program(https://blog.csdn.net/yaohucaizi/article/details/8863823), using the following two libraries:
org.apache.tools.zip.ZipOutputStream;
import org.apache.tools.zip.ZipEntry;
and the zip file is too large (4.5GB), so i cannot upload it.
I added some printing information of the decompression process, as follows:
[root@localhost opt]# unzip MyTxxx.zip
Archive: MyTxxx.zip
cover_add:beg:4793290554,end:4795054264
cover_add:beg:4795054264,end:4795054320
cover_add:beg:4795054320,end:4795054362
cover_within:val:0,ret=0
creating: scripts/
cover_within:val:56,ret=0
creating: scripts/ixx/
cover_within:val:117,ret=0
inflating: scripts/ixx/change_xxx.sh
cover_add:beg:117,end:761
cover_within:val:753,ret=1
error: invalid zip file with overlapped components (possible zip bomb)
btw:
I would like to ask: compressing some files using the zip command, will it produce overlapped entries?
47b3cea
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
47b3cea
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hello
“Detect and reject a zip bomb using overlapped entries.”
I didn't figure out whether the overlapping entries refer to the local header or the central directory?
Can I add some parameters to set the number of overlapping entrie allowed, if it exceeds this threshold, it will no longer decompress. If so, please give me some suggestions and guidance.
47b3cea
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
4.5GB isn't too large. You could upload it to Google Drive, which is free up to 15 GB. And then share it with me (google at madler dot net).
47b3cea
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And make sure that you have also applied a later commit: 6d35183 that follows this one.
47b3cea
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
im sure I applied the commit: 6d35183
47b3cea
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These commits detect any zip file element overlapping with any other zip file element. A valid zip file will have no such overlaps.
Either your zip file is invalid in this way, or there is a bug in my code to detect such overlaps. The only way I can distinguish between these two is to be able to examine your zip file.
47b3cea
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please run
unzip -v
and provide the output, so I can see the compile options. Thanks.47b3cea
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[root@localhost test]# ./unzip -v
......
UnZip special compilation options:
COPYRIGHT_CLEAN (PKZIP 0.9x unreducing method not supported)
SET_DIR_ATTRIB
SYMLINKS (symbolic links supported, if RTL and file system permit)
TIMESTAMP
UNIXBACKUP
USE_EF_UT_TIME
USE_UNSHRINK (PKZIP/Zip 1.x unshrinking method supported)
USE_DEFLATE64 (PKZIP 4.x Deflate64(tm) supported)
UNICODE_SUPPORT [wide-chars, char coding: UTF-8] (handle UTF-8 paths)
MBCS-support (multibyte character support, MB_CUR_MAX = 6)
LARGE_FILE_SUPPORT (large files over 2 GiB supported)
ZIP64_SUPPORT (archives using Zip64 for large files supported)
USE_BZIP2 (PKZIP 4.6+, using bzip2 lib version 1.0.6, 6-Sept-2010)
VMS_TEXT_CONV
[decryption, version 2.11 of 05 Jan 2007]
UnZip and ZipInfo environment options:
UNZIP: [none]
UNZIPOPT: [none]
ZIPINFO: [none]
ZIPINFOOPT: [none]
47b3cea
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good. (Was checking for large file and zip64.)
Please do an
unzip -Zv MyTxxx.zip
and provide the output.Have you been able to upload the file to google drive?
47b3cea
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
#unzip -Zv MyTxxx.zip
...
Central directory entry #11618:
There are an extra 16 bytes preceding this file.
mxx/xx/Eqxx_en.xml
offset of local header from start of archive: 4793288797
(000000011DB3C85Dh) bytes
file system or operating system of origin: Unix
version of encoding software: 4.5
minimum file system compatibility required: MS-DOS, OS/2 or NT FAT
minimum software version required to extract: 4.5
compression method: deflated
compression sub-type (deflation): normal
file security status: not encrypted
extended local header: yes
file last modified on (DOS date/time): 2019 Sep 26 20:34:14
32-bit CRC value (hex): 8d367b7f
compressed size: 1667 bytes
uncompressed size: 18092 bytes
length of filename: 44 characters
length of extra field: 12 bytes
length of file comment: 0 characters
disk number on which file begins: disk 1
apparent file type: binary
Unix file attributes (001130 octal): ?--x-wx--T
MS-DOS file attributes (01 hex): read-only
and I run
unzip -v xxx.zip
, it list all 11618 files info OK.but for some reason, I can not upload the zip file to google drive.
47b3cea
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you provide all of the output of
unzip -Zv
? Perhaps you would be able to put at least that on google drive.47b3cea
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also I pushed a few more commits yesterday that you could try. I suspect that those won't make a difference to what you're seeing, but it's worth a try.
47b3cea
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @madler ,
I also recently got this similar issue with https://downloads.rclone.org/rclone-current-linux-amd64.zip over alpine.
output of
unzip -Zv
: