archive/zip: reject certain invalid archives

[MichaelTJones]

This is not a defect report, but a suggestion to review the Zip archive reader with this new and authoritative zipbomb treatise in mind:

https://www.bamsoftware.com/hacks/zipbomb/

[ianlancetaylor]

CC @dsnet

[dsnet]

Hi @MichaelTJones, thanks for the report. This attack depends on two properties (called "insights" in the paper).

I believe Go's zip.Writer refuses to encode zip archives like this. However, Go's zip.Reader is probably susceptible to both properties mentioned by insight 1 and 2 (I did not verify). Insight 2 seems to be obviously an invalid ZIP archive, and our reader should error on such files. However, it's not clear to me that the directly overlapping files of insight 1 is invalid. I personally believe this should be an error as well, but I'm concerned that there may be people actually depending on this property.

[MichaelTJones]

I read the whole paper. The other point was that in addition to specific defenses, a careful implementation needs limits on time and space. That seemed good to me--if the expansion passes 1 GiB and the user's not set a more liberal upper limit, then bail. I do not believe (but do not know for sure) that there are ever valid cases for overlapping file bodies. I'm thinking that this should either be prohibited or like file size limit, be an option that defaults to disabled.

…

On Wed, Jul 10, 2019 at 10:49 AM Joe Tsai ***@***.***> wrote: Hi @MichaelTJones <https://github.com/MichaelTJones>, thanks for the report. This attack depends on two properties (called "insights" in the paper). I believe Go's zip.Writer refuses to encode zip archives like this. However, Go's zip.Reader is probably susceptible to both properties mentioned by insight 1 and 2 (I did not verify). Insight 2 seems to be obviously an invalid ZIP archive, and our reader should error on such files. However, it's not clear to me that the directly overlapping files of insight 1 is invalid. I personally believe this should be an error as well, but I'm concerned that there may be people actually depending on this property. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#33026?email_source=notifications&email_token=AB4DFJKRBT2ATW2QVG3JJ6DP6YOMJA5CNFSM4H7QSYG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZUHECI#issuecomment-510161417>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB4DFJLFRXAUN7X52OB22H3P6YOMJANCNFSM4H7QSYGQ> .

-- *Michael T. Jonesmichael.jones@gmail.com <michael.jones@gmail.com>*

[FiloSottile]

I read that article when it came out, and while it's extremely interesting in its techniques, I agree with the point it makes that unpacking archives with untrusted sources requires CPU, time and memory limits.

We should fix archive/zip if it accepts invalid archives, but I don't think we can make it safe to run on arbitrary archives without resource limits. Offering an API to easily apply limits on the unpacking process would be interesting, but a separate issue and definitely not for Go 1.13.

Aside from the insights, the table also points out other invalid zips that we decode, so we should probably also fix those.

/cc @rsc since he's credited on that article, and might have relevant insight.

[MichaelTJones]

I'll only add that lately I've been processing an archive (RSC's Go Corpus, split, bundled, compressed, an archived that is 752 MB when decompressed, so the defaults might be pretty big -- a few seconds, a few GB, etc. -- rather than cater to MIME-type decompression sizes.

…

On Wed, Jul 10, 2019 at 12:07 PM Filippo Valsorda ***@***.***> wrote: I read that article when it came out, and while it's extremely interesting in its techniques, I agree with the point it makes that unpacking archives with untrusted sources requires CPU, time and memory limits. We should fix archive/zip if it accepts invalid archives, but I don't think we can make it safe to run on arbitrary archives without resource limits. Offering an API to easily apply limits on the unpacking process would be interesting, but a separate issue and definitely not for Go 1.13. Aside from the insights, the table also points out other invalid zips that we decode, so we should probably also fix those. /cc @rsc <https://github.com/rsc> since he's credited on that article, and might have relevant insight. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#33026?email_source=notifications&email_token=AB4DFJMWNDMTS4TYA3KESPDP6YXOXA5CNFSM4H7QSYG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZUNYWA#issuecomment-510188632>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB4DFJND6DLJKKPH74LGREDP6YXOXANCNFSM4H7QSYGQ> .

-- *Michael T. Jonesmichael.jones@gmail.com <michael.jones@gmail.com>*

[qbradq]

Is anyone working on this? If not I'd like to take it.

[MichaelTJones]

There is more here: #33036 It is not necessary to address it everywhere at once by the same mechanism, but it does seem important to approach zip decoding as an instance of the general issue. (As in maybe there is a "limit" type passed as an optional argument, limit having setters & setters but opaque contents, and thus the ability to change implementation details later as more kinds of encoding are protected this way.) That's just an idea for consideration. Nobody "has" this at present.

…

On Mon, Jul 15, 2019 at 5:29 PM Norman B. Lancaster < ***@***.***> wrote: Is anyone working on this? If not I'd like to take it. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#33026?email_source=notifications&email_token=AB4DFJMEA3JMRLPJZV2FE4LP7UI6FA5CNFSM4H7QSYG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZ7KTCY#issuecomment-511617419>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB4DFJOPPNRGHSRJNSA73LDP7UI6FANCNFSM4H7QSYGQ> .

-- *Michael T. Jonesmichael.jones@gmail.com <michael.jones@gmail.com>*

[AxbB36]

A couple of things to watch out for here -- reports of in-the-wild zip files that could no longer be processed after Debian merged a patch to reject overlapping files (#931433: unzip: CVE-2019-13232).

• #931895: unzip: zip bomb false positives in Java ecosystem
  The problem here seems to be duplicate central directory headers that point to the same local file header, which shows up as overlapping files. See particularly Message #73. See also Gradle jars "grossly invalid zip files" that trigger zip bomb detection [CVE-2019-13232] gradle/gradle#9990.
• #932404: firefox-esr, FTBFS "possible zip bomb".
  The issue here is Firefox's omni.ja file, which uses a nonstandard zip layout with the central directory near the beginning, not the end. This has caused plenty of problems over the years (e.g. Mozilla #595473, #605524, #775568), but archive/zip can nevertheless handle it currently.