GZipMiddleware does not flush compressed bytes for streaming response chunk

[janbjorge]

GZipMiddleware never calls gzip_file.flush() for intermediate streaming chunks (more_body=True). The zlib compressor holds output in its internal buffer and only releases bytes when its ~32KB threshold is exceeded or the stream closes. This effectively destroys progressive delivery for streaming responses.

Minimal Reproducible Example

from starlette.middleware.gzip import GZipResponder

# GZipResponder wraps Python's gzip.GzipFile which uses zlib internally.
# zlib buffers compressed output until ~32KB accumulates or the stream closes.
# For streaming responses, this means clients receive nothing until the end.
responder = GZipResponder(app=None, minimum_size=500)

# Simulate 10 streaming chunks (more_body=True)
for i in range(10):
    body = responder.apply_compression(b"x" * 4000, more_body=True)
    # Without flush: most chunks produce 0 bytes
    # With flush: every chunk produces compressed output immediately
    print(f"Chunk {i}: {len(body)} compressed bytes")

# Final chunk closes the gzip stream
final = responder.apply_compression(b"x" * 4000, more_body=False)
print(f"Final:    {len(final)} compressed bytes")

Output:

Chunk 0: 10 compressed bytes
Chunk 1: 0 compressed bytes
Chunk 2: 0 compressed bytes
Chunk 3: 0 compressed bytes
Chunk 4: 0 compressed bytes
Chunk 5: 0 compressed bytes
Chunk 6: 0 compressed bytes
Chunk 7: 0 compressed bytes
Chunk 8: 0 compressed bytes
Chunk 9: 0 compressed bytes
Final:    68 compressed bytes

Chunk 0 emits only the 10-byte gzip header. Chunks 1-9 produce 0 bytes — all compressed data is released only when the stream closes.

Proposed fix

Add self.gzip_file.flush() for intermediate chunks. GzipFile.flush() issues a zlib Z_SYNC_FLUSH, which forces all pending compressed bytes to be emitted:

def apply_compression(self, body: bytes, *, more_body: bool) -> bytes:
    self.gzip_file.write(body)
    if not more_body:
        self.gzip_file.close()
    else:
        self.gzip_file.flush()

    body = self.gzip_buffer.getvalue()
    self.gzip_buffer.seek(0)
    self.gzip_buffer.truncate()
    return body

References

• zlib manual — Z_SYNC_FLUSH: "all pending output is flushed to the output buffer and the output is aligned on a byte boundary, so that the decompressor can get all input data available so far"
• Python gzip docs / Python zlib docs — GzipFile.flush() defaults to zlib_mode=zlib.Z_SYNC_FLUSH

[Kludex]

MRE with Starlette please

[janbjorge]

Updated the initial post

[Kludex]

• Seems the same proposal as flush() after writing to gzip_file #2753.

[janbjorge]

They are a releated, however we use GZipMiddleware with streaming responses; not SSE, but large-ish datasets (CSV exports, NDJSON) that we stream down to clients. We hit the buffering issue described in this PR: compressed bytes are stuck in zlib's buffer until ~32KB fills or the stream closes, so clients sit waiting instead of processing data as it arrives.

Would you consider adding an opt-in flag like streaming_flush=True? That way the decision is ours to make rather than the middleware making it for us. Default behavior stays exactly the same.