ssl.SSLError: EOF occurred in violation of protocol as a result of zlib compression. #84

Closed
RishiRamraj opened this Issue Nov 22, 2012 · 1 comment

Projects

None yet

2 participants

@RishiRamraj
Contributor

Hello All,

I stumbled across this bug when using webob in a wsgiref server, to serve gzip encoded content. I think I've found the problem and I'll issue a patch and pull request shortly (with tests). Bear in mind that this is the first time I've parsed this code, so please correct my reasoning below if necessary.

I am currently running Python3.2 in Ubuntu 12.03. I haven't tested in any other versions. I'm running webob inside a wsgiref server that is serving https using this recipe:

http://tibit.com/code/wsgissl.py

I'm currently using SSLV3 in the SSL version and it does not seem to like when the _app_iter property of the Response object yields an empty byte string; it's a violation of protocol. In response.py we have this bit of code:

yield _gzip_header
for item in app_iter:
    size += len(item)
    crc = zlib.crc32(item, crc) & 0xffffffff
    yield compress.compress(item)
yield compress.flush()
yield struct.pack("<2L", crc, size & 0xffffffff)

Unfortunately, both compress.compress(item) and yield compress.flush() may yield zero byte strings, depending on the input. I dug into the zlib module to find out exactly what it's doing. The compression object uses this function call to compress the input:

err = deflate(&(self->zst), Z_NO_FLUSH);

I looked at the documentation for deflate and found this:

Normally the parameter flush is set to Z_NO_FLUSH, which allows deflate to
decide how much data to accumulate before producing output, in order to
maximize compression.

It seems the compression algorithm accumulates data, and when the buffer is full, yields data to the caller. Otherwise it yields an empty byte string. Here's a simple example illustrating this behaviour:

import zlib
compress = zlib.compressobj(9, zlib.DEFLATED, -zlib.MAX_WBITS,
                            zlib.DEF_MEM_LEVEL, 0)

test = ['this is a test']

for item in test:
    result = compress.compress(item)         
    if not result:
        print('compress yielded no data')

result = compress.flush()
if not result:
    print('flush yielded no data')

The patch I've come up with is along the lines of:

yield _gzip_header
for item in app_iter:
    size += len(item)
    crc = zlib.crc32(item, crc) & 0xffffffff
    result = compress.compress(item)
    if result:
        yield result
result = compress.flush()
if result:
    yield result
yield struct.pack("<2L", crc, size & 0xffffffff)

Does this issue seem legitimate?

@twillis twillis pushed a commit to Batterii/webob that referenced this issue Mar 23, 2013
Tom Willis bring test coverage back to 100% for pull request 85 which fixes bug #84
1270039
@mcdonc
Member
mcdonc commented Aug 10, 2013

Looks like the merging of #85 closed this.

@mcdonc mcdonc closed this Aug 10, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment