deflate decompress error #23

Closed
lilydjwg opened this Issue Oct 27, 2012 · 4 comments

Comments

Projects
None yet
2 participants

I got this when decompressing a deflate-encoded response from http://baike.baidu.com/view/262.htm :

Exception zlib.error: 'Error -3 while decompressing data: incorrect header check' in 'http_parser.parser.on_body_cb' ignored

According to this, -zlib.MAX_WBITS should be passed to zlib.decompressobj. And it solved my problem.

Owner

benoitc commented Feb 9, 2013

can you provide a patch?

Owner

benoitc commented Feb 9, 2013

mmm in deflate, you shouldn't get a gzipped response which is all the point of 16+zlib.MAX_WBITS . Maybe smth wrong in the way they handle deflate.

Here's what I've changed:

diff --git a/http_parser/parser.pyx b/http_parser/parser.pyx
index 4ec9481..835842c 100644
--- a/http_parser/parser.pyx
+++ b/http_parser/parser.pyx
@@ -109,7 +109,7 @@ cdef int on_headers_complete_cb(http_parser *parser):
             res.decompressobj = zlib.decompressobj(16+zlib.MAX_WBITS)
             del res.headers['content-encoding']
         elif encoding == 'deflate':
-            res.decompressobj = zlib.decompressobj()
+            res.decompressobj = zlib.decompressobj(-zlib.MAX_WBITS)
             del res.headers['content-encoding']
         else:
             res.decompress = False

It seems there are two kinds of deflate in real worldlink. I wonder if there is any chance to handle them both?

Owner

benoitc commented Feb 10, 2013

Maybe a better fix would be to check if an exception is raised the first time we add some data and recreate the object . I will propose a patch later today.

benoitc closed this in a541efb Aug 30, 2013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment