Add transparent compression handling in the Response struct.

[szank]

Hi,
I was wondering if adding transparent compression handling in the code is feasible.
Currently we check if the "Content-Encoding" header have a "gzip" value and decompress the data ourselves.
I might do it myself if I have some free time on my hands, but I was wondering if this change would be even accepted

[valyala]

Currently the client doesn't set Accept-Encoding headers to gzip or deflate, so conforming server should never send compressed response with 'Content-Encoding' response header.

Transparent compression may be added to the client as opt-in setting in Client and HostClient. EnableCompression is a good name for this setting. You may create pull request for this feature.

Enabling transparent compression in fasthttp client by default is no go, since default compression implementation in Go is slooow ;)

[szank]

My idea was to check for the 'Content-Encoding' header in the response, and decompress the body if the header was present.
I didn't indent do modify the request sending path, only receiving. We could enable this behaviour by default, and disable it by setting 'AutoDecompress' to false in the client/response, where the value from the response would take precedence.

Now, We could add a EnableCompression parameter also, to transparently compress the requests, but this is a story for another pull request.

In our use case, we use haproxy to compress the responses sent from the server ( We do set 'Content-Encoding' in the request header explicitly), and then check for the 'Content-Encoding' in the response.

[valyala]

Nice idea! I'll try implementing it on the next week.

[valyala]

@szank , I decided adding helper functions for body compression instead of transparent compression. This allows using response body compression only when needed.

See Response.BodyGunzip and Response.BodyInflate.

[valyala]

Added CompressHandler for transparent response compression on server side.

[dibu28]

There is github.com/klauspost/compress a fully gzip compatible drop in replacement for go standard gzip/zip/zlib packages.
Claimed to be up to 3x faster for data compression.
Description and benchmarks by the author:
http://blog.klauspost.com/go-gzipdeflate-benchmarks/
http://blog.klauspost.com/gzip-performance-for-go-webservers/

pgzip Go parallel gzip compression/decompression. This is a fully gzip compatible drop in replacement for "compress/gzip". Faster for big files more than 1MB.

cgzip wraps zlib using cgo.

[sschepens]

What about transparent decompression of incoming requests to server?

[valyala]

Just added Request.BodyGunzip helper.

Transparent compression / decompression won't be implemented in fasthttp, since this may be abused by specially crafted malicious requests / responses, which may eat a lot of CPU and memory resources. So fasthttp users must manually compress / decompress request / response bodies when needed.

[valyala]

I think this issue may be closed now. Feel free creating new issues for unresolved topics mentioned above.

[sschepens]

Thanks! Isn't BodyInflate() missing?