Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Suggestion to implement transparent gzip decompression. #42

Closed
Teshootub7 opened this issue Nov 17, 2010 · 2 comments
Closed

Suggestion to implement transparent gzip decompression. #42

Teshootub7 opened this issue Nov 17, 2010 · 2 comments
Labels
BUG

Comments

@Teshootub7
Copy link

Hi,

I think it would be great if httpclient support gzip decompression because it is
widely used on many websites and reduces network traffics dramatically.
So I made a small patch to support gzip decompression on httpclient.

https://gist.github.com/702845

This patch adds a "transparent_gzip_decompression" attribute to HTTPClient. It is set to false by default for backward compatibility. With setting true to this attribute:

  • "Accept-Encoding: gzip,deflate" will be added to every request headers.
  • Decompress the content body automatically if the response header has "Content-Encoding: gzip OR x-gzip OR deflate".

Test cases for this change is also included in the patch.

Regards,
@Teshootub7
Copy link
Author

Additional info:

Sorry, I forgot to provide an example...
All you need to use "transparent gzip decompression" is done by:

clnt.transparent_gzip_decompression = true

Then you can retrieve a document as usual.

res = clnt.get("http://www.debian.org/")

The document is transferred with gzip compressed.

res.header["Content-Encoding"]
=> ["gzip"]
res.header["Content-Length"]
=> ["4422"]
res.content.length
=> 13942

And you can read the content without knowing it is compressed or not.

puts res.content
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html lang="en">
<head>
  <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
  <title>Debian -- The Universal Operating System </title>
....

Regards,

@nahi
Copy link
Owner

nahi commented Dec 12, 2010

Added transparent_gzip_decompression property. closed by 95964f1.

Applied a patch from Teshootub7. With setting
HTTPClient#transparent_gzip_decompression = true, HTTPClient tries to do
gzip/deflate content transmission. Following explanation is by
Teshootub7.

All you need to use "transparent gzip decompression" is done by:

clnt.transparent_gzip_decompression = true

Then you can retrieve a document as usual.

res = clnt.get("http://www.debian.org/")

The document is transferred with gzip compressed.

res.header["Content-Encoding"]
=> ["gzip"]
res.header["Content-Length"]
=> ["4422"]
res.content.length
=> 13942

And you can read the content without knowing it is compressed or not.

apepper added a commit to apepper/faraday that referenced this issue Jan 14, 2015
@apepper
httpclient: Enable gzip compression.
c877521 
Motivation: httpclient support gzip compression since late 2010 (see nahi/httpclient#42 ).
@apepper apepper mentioned this issue Jan 14, 2015
Merged
@milgner milgner mentioned this issue Nov 13, 2015
Closed
@shivanshgaur shivanshgaur mentioned this issue May 20, 2018
Closed
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
BUG
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@nahi @Teshootub7