Expose zlib limited output buffer size functionality.

[seriyps]

This functionality may be useful for compressed streams with high compression ratio (in case of gzip it may be up to x1000), when small amount of compressed input will produce large amount of uncompressed output. This may lead to DoS attacks, because server easily goes out of memory.

Example of such high compression ratio stream:

dd if=/dev/zero of=sparse.bin bs=1MB count=100 # 100mb of zeroes
gzip sparse.bin  # 95kb sparse.bin.gz
$ erl
> {ok, Compressed} = file:read_file("sparse.bin.gz"),
> 97082 = size(Compressed),
> Uncompressed = zlib:gunzip(Compressed),
> 100000000 = iolist_size(Uncompressed).

Imagine HTTP client, which download such Content-Encoding: gzip stream from remote WWW host.

So, suggested solution will look like this:

walk(Compressed, Handler) ->
    Z = zlib:open(),
    zlib:inflateInit(Z),
    % Limit single uncompressed chunk size to 512kb
    zlib:setBufSize(Z, 512 * 1024),
    loop(Z, Handler, zlib:inflateChunk(Z, Compressed)),
    zlib:inflateEnd(Z),
    zlib:close(Z).

loop(Z, Handler, {more, Uncompressed}) ->
    % iolist_size(Uncompressed) =< 512 kb
    Handler(Uncompressed),
    loop(Z, Handler, zlib:inflateChunk(Z));
loop(Z, Handler, Uncompressed) ->
    % iolist_size(Uncompressed) =< 512 kb
    Handler(Uncompressed).

And yes, it uses existing zlib:setBufSize/2 to configure output chunk size.

[OTP-Maintainer]

The summary line of the commit message is too long and/or ends with a "."
Make sure the whole message follows the guidelines here: https://github.com/erlang/otp/wiki/Writing-good-commit-messages.

Bad message: Expose zlib limited output buffer size functionality.

---

I am a script, I am not human

---

[seriyps]

Fixed

[nox]

You left a typo in the commit message: "functiomality". Nice feature, otherwise.

[seriyps]

@nox oups! Fixed.

[OTP-Maintainer]

Patch has passed first testings and has been assigned to be reviewed

---

I am a script, I am not human

---

[essen]

+1.

[bjorng]

Thanks, looks good, but I have a few comments about style. I have added my comments directly to the commit.

[seriyps]

Fixed

[OTP-Maintainer]

Patch has passed first testings and has been assigned to be reviewed

---

I am a script, I am not human

---

[essen]

Will this make it into OTP 18?

[proxyles]

@essen that is the plan. If it pass the tests

[mworrell]

Great, this patch is exactly what I need to support gzip decoding in Zotonic. 👍

[proxyles]

Merged to master