Directly using Nginx as a server without Uvicorn

[rafsaf]

Hi, as I understand it, I can implement fastapi application with (or at least I know such options):

• only Uvicorn
• Uvicorn behind Nginx
• Uvicorn behind Gunicorn
• (Uvicorn behind something else?)

I found out that one can use Nginx unit directly with FastAPI as described here in nginx unit docs. It works as expected. It seems that using only Nginx is slightly simpler for me (maybe somehow faster or slower too? why?) than Uvicorn + something, but I haven't found anything in docs/issues about using it directly and using Uvicorn alone isn't recommended in some cases (as far as i know), hence the questions.

The questions are

• what are the possible drawbacks to using this approach / why should I avoid using Nginx directly
• if there are no contraindications, what additional steps should I eventually take to make it work well
• what are the main differences between these solutions (Uvicorn vs Uvicorn + something vs Nginx only)

To provide any context on how I exactly implemented it with Nginx only, let's say in our application's root folder we have config.json in config folder and hello world fastapi app in app/main.py, then it might look like this:

Dockerfile

FROM nginx/unit:1.23.0-python3.9
COPY ./config/config.json /docker-entrypoint.d/config.json
RUN mkdir build
COPY . ./build
RUN apt update && apt install -y python3-pip                                  \
    && pip3 install -r /build/requirements.txt                               \
    && apt remove -y python3-pip                                              \
    && apt autoremove --purge -y                                              \
    && rm -rf /var/lib/apt/lists/* /etc/apt/sources.list.d/*.list
EXPOSE 80

config.json

{
    "listeners": {
        "*:80": {
            "pass": "applications/fastapi"
        }
    },

    "applications": {
        "fastapi": {
            "type": "python 3.9",
            "path": "/build/app/",
            "module": "main",
            "callable": "app"
        }
    }
}

[frankie567]

I wasn't aware of Nginx Unit, that's interesting!

First of all, let's clarify things a bit:

• FastAPI (thanks to its underlying server layer Starlette) implement the Python ASGI protocol. It's a standard to build web application with async support. It's the successor of WSGI, which is the standard protocol to build synchronous web applications.
• Uvicorn is an implementation of an ASGI server. Its role is to run ASGI apps (like FastAPI) and expose an HTTP server.
  • There are others ASGI server implementations out there, like Daphne or Hypercorn.
• Gunicorn, at its base, is a WSGI server implementation, widely used with Flask or Django for example.
  • However, it has interesting features to manage and monitor server processes, which may make it more resilient in production.
  • That's why it's recommended to run FastAPI apps with Gunicorn configured with an Uvicorn worker, to make it work with ASGI protocol.
• I'm less aware of the implications of running Uvicorn behind Nginx, or even Uvicorn-Gunicorn-Nginx. I guess it allows you to have interesting rules to optimize static file serving or caching. I would say that Uvicorn-Gunicorn-Nginx may be a bit useless.

Now, for "Nginx Unit". From what I understand, it's an Nginx module implementing its own version of ASGI; so it comes "instead of" Uvicorn, it's not just Nginx on its own. Actually, it seems that you can even configure Nginx (the full HTTP server) in front of Nginx Unit (the app server): https://unit.nginx.org/howto/integration/

I see that they have an implementation of ASGI in pure C. I would say that it should work quite well, after all, Nginx has proven to be very efficient for years, so I guess the people behind know their stuff.

The difference could be in terms of performance; and the only way to find out which one is better is to perform benchmarks. I didn't find existing for ASGI, that could be interesting to do.

[Kludex]

The difference could be in terms of performance; and the only way to find out which one is better is to perform benchmarks. I didn't find existing for ASGI, that could be interesting to do.

You can try changing the event loop and see how it behaves:

import uvloop

asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())

[Faylixe]

Wow, I wasn't aware of that either, that would be great to benchmark all existing process manager solution indeed, I personally ended up using Hypercorn with uvloop which is actually blazing fast and support heavy load (and also this is actually the only solution that is supporting HTTP/2).

[raphaelauv]

@Faylixe have a fair point HTTP/2 is not yet part of nginx unit nginx/unit#388

[synodriver]

I have tested nginx unit using wrk. It's around 20% faster than uvicorn with a real application that queries sqlite, and 40% faster than hypercorn under the same condition. Due to it's high throughput, it may be the future of asgi server. As for http2, uvicorn does not support it either. Hypercorn do. So if you need extreme fast servers you can try nginx unit, but if you want more asgi's feature you may try hypercorn

[Kludex]

Can I see the benchmarks?

[synodriver]

Sure. The follwing is nginx unit, uvicorn and hypercorn.

[Kludex]

I mean the setup, which commands did you use, on what machines, what endpoints, etc...

[synodriver]

NGINX unit keeps high throughput under heavy load.

[synodriver]

I mean the setup, which commands did you use, on what machines, what endpoints, etc...

I would create a new repo later...

[rafsaf]

I don't see how it's related to either my original question (in which, by the way, I did provide an example of sample implementation using Nginx Unit working just fine even after few months and a link to docs - nobody really asked for a direct help or examples but clarification of things) or mentioned benchmarks.

What about benchmarks - I would also love to see the whole setup of @synodriver (or made by anyone). Stuff like that even with careful design may have little to do with reality, there are just so many config options (of servers), VM implementation details, FastAPI details to consider (sync/async endpoins for example) etc.

[synodriver]

I don't see how it's related to either my original question (in which, by the way, I did provide an example of sample implementation using Nginx Unit working just fine even after few months and a link to docs - nobody really asked for a direct help or examples but clarification of things) or mentioned benchmarks.

What about benchmarks - I would also love to see the whole setup of @synodriver (or made by anyone). Stuff like that even with careful design may have little to do with reality, there are just so many config options (of servers), VM implementation details, FastAPI details to consider (sync/async endpoins for example) etc.

Hi, here is the repo for bench mark.

[ghandic]

Thanks for posting your research code up @synodriver ! Could you add a raw md table to your repo README for direct comparison?

[Kludex]

Thanks! @synodriver