RuntimeError: Stream consumed when reading body with startlette (bug since 1.9.10)

[sevaho]

How do you use Sentry?

Sentry Saas (sentry.io)

Version

1.9.10

Steps to Reproduce

Server:

import sentry_sdk

from fastapi import FastAPI, Request

sentry_sdk.init()

app = FastAPI()


@app.post("/")
async def index(request: Request):
    r = await request.body()
    return {"Hello": "World"}

Run with uvicorn:

uvicorn app:app --port 5000

As client do:

#! /usr/bin/python
import httpx

import asyncio


async def main():
    image = b"\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\x01\x00\x00\x00\x01\x08\x06\x00\x00\x00\x1f\x15\xc4\x89\x00\x00\x00\rIDATx\xdac\xfc\xcf\xf0\xbf\x1e\x00\x06\x83\x02\x7f\x94\xad\xd0\xeb\x00\x00\x00\x00IEND\xaeB`\x82"
    files = {"file": ("test.png", image)}
    async with httpx.AsyncClient() as client:
        r = await client.post(
            "http://localhost:5000/",
            files=files,
        )
        print(r)


asyncio.run(main())

Expected Result

To succeed.

Actual Result

ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 404, in run_asgi
result = await app( # type: ignore[func-returns-value]
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in call
return await self.app(scope, receive, send)
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/fastapi/applications.py", line 270, in call
await super().call(scope, receive, send)
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 293, in _sentry_patched_asgi_app
return await middleware(scope, receive, send)
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py", line 138, in _run_asgi3
return await self._run_app(scope, lambda: self.app(scope, receive, send))
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py", line 187, in _run_app
raise exc from None
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py", line 182, in _run_app
return await callback()
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/starlette/applications.py", line 124, in call
await self.middleware_stack(scope, receive, send)
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 98, in _create_span_call
await old_call(*args, **kwargs)
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in call
raise exc
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in call
await self.app(scope, receive, _send)
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 191, in _sentry_exceptionmiddleware_call
await old_call(self, scope, receive, send)
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 98, in _create_span_call
await old_call(*args, **kwargs)
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in call
raise exc
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in call
await self.app(scope, receive, sender)
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 98, in _create_span_call
await old_call(*args, **kwargs)
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in call
raise e
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in call
await self.app(scope, receive, send)
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/starlette/routing.py", line 706, in call
await route.handle(scope, receive, send)
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle
await self.app(scope, receive, send)
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/starlette/routing.py", line 66, in app
response = await func(request)
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/sentry_sdk/integrations/fastapi.py", line 106, in _sentry_app
return await old_app(*args, **kwargs)
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/fastapi/routing.py", line 231, in app
raw_response = await run_endpoint_function(
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/fastapi/routing.py", line 160, in run_endpoint_function
return await dependant.call(**values)
File "/home/sevaho/gitlab/centurion/./app.py", line 14, in read_root
r = await request.body()
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/starlette/requests.py", line 234, in body
async for chunk in self.stream():
File "/home/sevaho/.cache/pypoetry/virtualenvs/centurion-X1KYOsH6-py3.10/lib/python3.10/site-packages/starlette/requests.py", line 215, in stream
raise RuntimeError("Stream consumed")
RuntimeError: Stream consumed

[antonpirker]

Hey @sevaho

Thanks for reporting this. We will have a look when there is time. (Honestly I thought I have fixed this already...)

[sevaho]

Thanks @antonpirker !

Looks like there is more to it as well.

@app.post("/")
async def index(request: Request, file: UploadFile = File(...)):
    print(file.filename)
    f = await file.read()
    print(f)
    return ""

Whatever file you pass, the content for await file.read() is always empty due to sentry-sdk absorbing it?

[mociepka]

I am facing the same issue in one of the projects I am involved. I can share just traceback:

If you need more details, I am happy to help.

[mecampbellsoup]

Downgrading to ~1.8 for the time being. Saw this bug on version 1.10.1.

[ranjith19]

I had about 20k errors caused by this since yesterday and I was off. Downgraded the library once I figured this out. Lost a lot of API calls.

Is there a way to reset quota on sentry because this bug was essentially caused due to sentry library issue?

[antonpirker]

Hello! I will have a look at this first thing on Monday!

[antonpirker]

I was (unplanned so) afk Monday and Tuesday, so I am looking into this now.

[antonpirker]

This is happening because of some inconsistencies on how Starlette (the foundation of FastAPI) is dealing with request data:

I am now investigating if this should be fixed in Starlette/FastAPI or in Sentry:
encode/starlette#1933