Ray Serve with Fastapi is a lot (10x) slower than plain fastapi

[githuberj]

What happened + What you expected to happen

I have an issue or maybe just a wrong expectation while working with ray serve .
I thought it might be able to run our whole fastapi application but it looks like it might be 10x slower that a normal fastapi application. Therefore I did this small benchmark.
Maybe the benchmark is too simplistic. Could you provide a more comprehensive example then?
I would just like to have our expectations set: Is it a bad idea to put general application logic into Ray?

Versions / Dependencies

ray[serve] == 2.32.0
fastapi == 0.111.1

Reproduction script

I used docker to test it and provided a zip file.
benchmark.zip

Issue Severity

Low: It annoys or frustrates me.

[Superskyyy]

Hello, see this related issue, but in general Ray serve is not expected to deliver high qps like normal FastAPI as it has a single point of ingress (proxy actor) that brings additional overhead, but I expect the overhead can be lowered in the future.

[githuberj]

Thank you for the response. I will close the issue.