GZip Middleware Throws "Response content longer than Content-Length" when given a 304 empty-body response

[nicknotfun]

First Check

• I added a very descriptive title to this issue.
• I used the GitHub search to find a similar issue and didn't find it.
• I searched the FastAPI documentation, with the integrated search.
• I already searched in Google "How to X in FastAPI" and didn't find any information.
• I already read and followed all the tutorial in the docs and didn't find an answer.
• I already checked if it is not related to FastAPI but to Pydantic.
• I already checked if it is not related to FastAPI but to Swagger UI.
• I already checked if it is not related to FastAPI but to ReDoc.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from typing import Awaitable, Callable

from fastapi import FastAPI
from fastapi.middleware.gzip import GZipMiddleware
from starlette.requests import Request

app = FastAPI()


@app.middleware("http")
async def noop_middleware(request: Request, call_next: Callable[[Request], Awaitable]):
    return await call_next(request)


@app.get("/", status_code=304)
def youve_seen_this_before():
    pass


app.add_middleware(GZipMiddleware)

Description

curl -sH "Accept-Encoding: gzip" localhost:8000

This will reliably:
a) Give the expected 304 empty response.
b) Show a stack trace "Response content longer than Content-Length"

The root cause I believe is the base middleware (in this example the noop one) creates a StreamingResponse for what is ultimately an empty response, it consists of two streaming chunks ("", more=True), ("", more=False); which bypasses the minimum length check in the GZip middleware.

GZip then compresses the 'nothing' resulting in the GZip minimum boilerplate, but an expected content length of 0 then throws. This is post-response so the client wont see the error but the server will.

Operating System

Linux, macOS

Operating System Details

Don't think this is relevant.

FastAPI Version

0.63.0

Python Version

3.8.10

Additional Context

This is actually the root cause of issue: #2818

In the case of that issue the trigger is the StaticFiles mount has not-modified support, and on getting a non-modified static resource returns a 304. I suspect the original issue reporter stopped running into it either as a by-product of not getting 304 as often, or alternatively changing their middleware so as not to create the StreamingResponse.

[nicknotfun]

FYI, I believe this reproduces on any empty response (confirmed on 204 as well, 200 works as it technically has a body)

[NickKush]

I have the same issue for a year already. I guess it's a Starlette problem

[levrik]

Experiencing the same issue with a 204 response. Which has an empty body by design.

[hiranp]

We encountered this problem when developing our internal application, when we implemented both Brotli and GZIP compression. We tested the following fix using several use cases.

encode/starlette#1579

[tiangolo]

This can be reproduced even without the GzipMiddleware:

from fastapi import FastAPI

app = FastAPI()


@app.get("/", status_code=204)
def endpoint():
    print("Oh, hi mark")

More details in this Starlette issue: encode/starlette#1764

I'm fixing it on Starlette here: encode/starlette#1765

And I'm fixing it on FastAPI here: #5145

It will be available in the next version, released in some hours, FastAPI 0.79.0 🎉

[levrik]

@tiangolo I just tried FastAPI 0.79.0 but sadly the issue is still happening, at least with Gzip middleware enabled.
It never happened with Gzip middleware disabled for me so I can't say if it was fixed outside.

[Yolley]

@tiangolo I think we need to update starlette version in FastAPI dependencies, the issue with GZip Middleware was fixed in 0.20.1 encode/starlette#1579

Currently fix provided by #5145 does not work if GZip Middleware is used.

[Kludex]

This can be reproduced even without the GzipMiddleware:

from fastapi import FastAPI

app = FastAPI()


@app.get("/", status_code=204)
def endpoint():
    print("Oh, hi mark")

More details in this Starlette issue: encode/starlette#1764

I'm fixing it on Starlette here: encode/starlette#1765

And I'm fixing it on FastAPI here: #5145

It will be available in the next version, released in some hours, FastAPI 0.79.0 tada

Those 2 are different issues.

[Kludex]

@tiangolo I mean, those are different issues, but both were fixed.

[tiangolo]

Ah! Great, thanks @Kludex ! 🚀

[github-actions]

Assuming the original need was handled, this will be automatically closed now. But feel free to add more comments or create new issues or PRs.