This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
BrokenResourceError #4041
Comments
|
This looks like a starlette issue: encode/starlette#1284 |
Operating SystemLinux FastAPI Version0.70.0 Python Version3.9.7 |
|
I have the same problem, this is my code: main.py # -*- coding: utf-8 -*-
from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse
from fastapi.encoders import jsonable_encoder
from pony.orm import *
from models_pony.name_basics import NameBasics as pn_name_basics
api = FastAPI()
@api.middleware('http')
async def process_request(request: Request, call_next):
return await call_next(request)
@api.get('/namebasics/id/{id}')
async def get_namebasic_by_id(id:int):
with db_session:
result = (pn_name_basics[id]).to_dict()
return JSONResponse(result)
@api.get('/namebasics/ncost/{ncost}')
async def get_namebasic_by_ncost(ncost:str):
with db_session:
result = (pn_name_basics.get(ncost=ncost)).to_dict()
return JSONResponse(result)
@api.get('/namebasics/year/{year}')
async def get_namebasic_by_year(year:str):
with db_session:
results = select(p for p in pn_name_basics if p.birth_year == year)[:]
_results = [i.to_dict() for i in results]
_results = jsonable_encoder(_results)
return JSONResponse(_results)
@api.get('/namebasics/yearalive/{yearalive}')
async def get_namebasic_by_yearalive(yearalive:str):
with db_session:
results = select(p for p in pn_name_basics if p.birth_year == yearalive and p.death_year is None)[:]
_results = [i.to_dict() for i in results]
_results = jsonable_encoder(_results)
return JSONResponse(_results)Command terminal test Error Something that strikes me is that the application does not crash. The error is not displayed if I remove the middleware like issues encode/starlette/issues/1284 and the performance of the requests is better. Test 1 Test 2 Operating SystemLinux Operating System DetailsFastAPI Version0.70.0 Python Version3.6.12 Additional ContextWeb server: gunicorn 20.1.0 with worker class uvicorn.workers.UvicornWorker |
|
This is fixed in Starlette v0.17.0, hopefully FastAPI bumps starlette again on next release |
|
any progress? i have same problem too |
|
I hide the error message like this: async def _hide_anyio_BrokenResourceError(*args, **kwargs):
# hide `anyio.BrokenResourceError`
pass
@app.middleware('http')
async def before_each_endpoint(request: Request, call_next: RequestResponseEndpoint):
response = await call_next(request)
if await request.is_disconnected():
# hide `anyio.BrokenResourceError`
return _hide_anyio_BrokenResourceError
return responseBut I don’t know the hidden dangers of doing this |
|
@ixuuux I think hide the error message it isn't recommended, definitely something wrong happening and this could produce unexpected behaviors, so I do think it's dangerous IMHO. |
|
@alejosv Thank you, I am also worried about hiding error messages, so I don’t plan to do this. |
|
i catch same error with fastapi==0.70. is there any progress? |
|
I've catched the same error with fastapi==0.70 |
|
i am also facing the same problem, my sentry logs are filled with this error |
|
@raveenb I downgraded fastapi version to 0.68.2 and BrokenResourceError is "fixed" |
|
@Actticus thanks for the suggestion, let me give it a try |
I think this can be closed now that fastapi 0.71.0 bumped Starlette to 0.17.1 . |
|
I have tested FastAPI v0.71.0 in my project and the BrokenResourceError is fixed. |
@havardthom are you use a middleware? |
Yes |
…ore details. Add middleware for handle requests without headers in request.
…ore details. Add middleware for handle requests without headers in request.
…ore details. Add middleware for handle requests without headers in request.
Still seeing this with Using the middleware db session pattern, similar to |
|
Still seeing this issue |
|
@prsangappa Could you check what version you have for anyio? I went to 3.5.0 now and so far so good (together with fastapi 0.72.0 and starlette 0.17.1). Asking because I saw the following change went into anyio 3.5.0, see agronholm/anyio@3877a80#diff-81debd08048a10458b2748d33fe4c0229c1697dc0faa027ccd6bc9d4262137ffR138 |
|
I have |
Nope, got another one |
|
Assuming the original need was handled, this will be automatically closed now. But feel free to add more comments or create new issues or PRs. |
Main motivation is to try to fix a 'BrokenResourceError' exception with FastAPI/starlette: tiangolo/fastapi#4041
|
The issue is still relevant. We are running FastApi using Operating SystemDebian GNU/Linux 11 (bullseye) - container FastAPI Version0.75.0 Other libraries
Python Version3.10.2 TracebackTraceback (most recent call last):
File "starlette/middleware/cors.py", line 84, in __call__
await self.app(scope, receive, send)
File "ddtrace/contrib/asgi/middleware.py", line 178, in __call__
reraise(exc_type, exc_val, exc_tb)
File "six.py", line 719, in reraise
raise value
File "ddtrace/contrib/asgi/middleware.py", line 173, in __call__
return await self.app(scope, receive, wrapped_send)
File "starlette/exceptions.py", line 82, in __call__
raise exc
File "starlette/exceptions.py", line 71, in __call__
await self.app(scope, receive, sender)
File "fastapi/middleware/asyncexitstack.py", line 21, in __call__
raise e
File "fastapi/middleware/asyncexitstack.py", line 18, in __call__
await self.app(scope, receive, send)
File "starlette/routing.py", line 656, in __call__
await route.handle(scope, receive, send)
File "starlette/routing.py", line 259, in handle
await self.app(scope, receive, send)
File "starlette/routing.py", line 64, in app
await response(scope, receive, send)
File "starlette/responses.py", line 156, in __call__
await send({"type": "http.response.body", "body": self.body})
File "starlette/exceptions.py", line 68, in sender
await send(message)
File "ddtrace/contrib/asgi/middleware.py", line 170, in wrapped_send
return await send(message)
File "anyio/streams/memory.py", line 205, in send
raise BrokenResourceError
anyio.BrokenResourceError
Traceback (most recent call last):
File "anyio/streams/memory.py", line 193, in send
self.send_nowait(item)
File "anyio/streams/memory.py", line 186, in send_nowait
raise WouldBlock
anyio.WouldBlockRunning the app outside of a docker container on a stand-alone Ubuntu 20.04 LTS server we never experienced this issue. |
|
I just got this issue as well for the first time, with fastapi 0.75.0. Maybe it's related : this issue occured only a few days after I set up sentry, with its ASGI middleware : https://docs.sentry.io/platforms/python/guides/asgi/ Here is how I integrated it in FastAPI : By the way, the exception occured in the |
|
I'm also getting this, very often when using the CPU and network in the same process. These thing eventually crash the process because they make health checks fail. All the middleware you see (like mid.py) just calls the next handler in the middlware chain. Python 3.9.10. |
|
@tiangolo I have a set of files that will reliably, and quickly reproduce an AnyIO stream error when using ContextMiddleware, but don't when ContextMiddleware is removed. The stack trace is not the same as the ones above -- this test setup raises Could this issue be reopened, as there seems to be a number of other people who are also running into this? |
|
We just discovered that replacing ContextMiddleware with RawContextMiddleware in my test fast_app.py makes the |
|
@liavkoren The problem being most of these inheritance:s are deep inside libraries, right? |
|
We switched our CorrelationIdPlugin to use RawContextMiddleware (which doesn't inherit from the starlette baseMiddleware) and deployed that to prod yesterday at 4pm. We haven't had any BrokenResourceErrors since: We have a different service which has never run into this issue, and it sets up middleware like so: More of a workaround than a fix (what the heck is RawContextMiddleware doing? How is it different from starlette baseMiddleware? Is this bad? Who knows! I looked at the source for both very briefly and decided that things were not dire enough that I needed to go down those rabbit holes right now.] However, it has silenced the BrokenResourceErrors coming out of our /health endpoint since 4pm yesterday, and all our tests still pass and no one has yelled at us yet so.. cool? |
same |
|
First thanks tiangolo and all others fastapi contributors for providing such a cool and valuable tool! I'm currently migrating a wsgi app to a full fastapi app and I'm also facing this BrokenResourceError. Here is a minimal repro: app.py import time
from fastapi import FastAPI, Request
from fastapi.middleware.wsgi import WSGIMiddleware
import fla
app = FastAPI()
@app.middleware("http")
async def middleware(request: Request, call_next):
return await call_next(request)
@app.get("/")
def read_root():
time.sleep(4)
return {"Hello": "World from fastpi route"}
app.mount("/wsgi", WSGIMiddleware(fla.app), name="flask")fla.py import time
from flask import Flask
app = Flask(__name__)
@app.route("/")
def hello_world():
time.sleep(4)
return "<p>Hello, World! from flask route</p>"client.py import requests
r = requests.get(f"http://127.0.0.1:8000/wsgi/", timeout=2)I run Finally here is libraries versions: The relevant part is combination of |
Hopefully it fixes #279. See tiangolo/fastapi#4041
* fix: 🐛 reserve 256M for the API and nginx pods The API service seems to need about 52M: ``` process_virtual_memory_bytes 7.59681024e+08 process_resident_memory_bytes 5.2195328e+07 ``` * fix: 🐛 remove the PrometheusMiddleware to reduce the RAM usage Hopefully it fixes #279. See tiangolo/fastapi#4041
|
@cbernet We are also seeing this issue when using Sentry.io. Did you end up coming up with a solution? |
|
@don4of4 we didn't try and find a solution as the problem is very rare. It only happened once since my first report. I can't really conclude about sentry. In fact, without sentry, it could be that we would still be unaware of this issue. So maybe sentry simply revealed this issue, and it's unrelated. |
|
I had this issue as well, but after updating the libraries to these versions it got much better: Now when the client tries to receive large volume of data (>400MB), it got a 502 Bad Gateway error while FAST API reports 200 on the log file in the server. I am using GZipMiddleware and returning json data. Have any of you had the same issue or can help me? Thanks! |
env
codefrom loguru import logger
logger.remove()
PATH = './x.log'
LEVEL = 'INFO'
logger.add(
PATH, level=LEVEL, rotation="00:00", backtrace=True, diagnose=True, enqueue=True, compression='tar.gz'
)errordesc |
|
Do you use class CustomMiddleware(BaseHTTPMiddleware:
passor indirectly like: @app.middleware('http')
def middlewarefunc():
passI found that most people above are experiencing this with that particular middleware, which actually uses MemoryStream of anyio. |
This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
and others
First Check
Commit to Help
Example Code
app.py
script.py
Description
If you trigger script.py, error will appear.
After upgrade fastapi to 0.70.0 In some requests in our application its apearing this error. So what we can do? What is problem?
Operating System
Linux
Operating System Details
No response
FastAPI Version
0.70.0
Python Version
3.8
Additional Context
No response
The text was updated successfully, but these errors were encountered: