Uvicorn cannot be shutdown programmatically

[Andrew-lll]

There is no documented way to shutdown uvicorn in python:

ex:

instance = uvicorn.run("example:app", host="127.0.0.1", port=5000, log_level="info")
instance.shutdown()

How do we shutdown uvicorn?

[florimondmanca]

Hi,

Not documented indeed, but a multithreaded approach should do…

import contextlib
import time
import threading
import uvicorn

class Server(uvicorn.Server):
    def install_signal_handlers(self):
        pass

    @contextlib.contextmanager
    def run_in_thread(self):
        thread = threading.Thread(target=self.run)
        thread.start()
        try:
            while not self.started:
                time.sleep(1e-3)
            yield
        finally:
            self.should_exit = True
            thread.join()

config = Config("example:app", host="127.0.0.1", port=5000, log_level="info")
server = Server(config=config)

with server.run_in_thread():
    # Server started.
    ...
# Server stopped.

Very handy to run a live test server locally using a pytest fixture…

# conftest.py
import pytest

@pytest.fixture(scope="session")
def server():
    server = ...
    with server.run_in_thread():
        yield

[Andrew-lll]

I do appreciate the reply/code, but:

Why is this not built in and documented?
Why do I have to implement this code?

It's a webserver. Run_in_thread() and shutdown() are core functionality...am I wrong?

[plazmakeks]

I'm following @florimondmanca s approach to run pact tests on the provider side. However when using this i get the following error from uvicorn's Server.run method:

RuntimeError: There is no current event loop in thread 'Thread-7'.

Any idea on that? Happens with python 3.6.9 as well as 3.7.7.

another edit:

I got it to work using

config = Config("example:app", host="127.0.0.1", port=5000, log_level="info", loop="asyncio")

[Elijas]

I do appreciate the reply/code, but:

Why is this not built in and documented?
Why do I have to implement this code?

It's a webserver. Run_in_thread() and shutdown() are core functionality...am I wrong?

Fully agreed, this question of how to run and stop unicorn pops up on stack overflow as well, it's quite obscure as it is now

[florimondmanca]

I appreciate the feedback here, but then this brings the question — what would folks expect the usage API to be for something like this?

I think a nice approach for allowing a programmatic shutdown of Uvicorn would be to exit the space of .run() and embrace concurrency, that is the async serve() method. It's actually super easy to spin up a concurrent task, and then clean it up whenever the server isn't needed anymore. (We could also very well make it easier to do this multithreaded thing, and having with server.run_in_thread() built in, or something.)

But then there's the question of A/ waiting for the server to startup (without having to reinvent the wheel each time), and B/ triggering the server shutdown and have it clean up gracefully (straight up cancelling a task is very... brutal).

Personally, I'd love this kind of API:

async with open_task_group() as tg:
    tg.start_soon(server.serve)
    await server.wait_started()
    # ...
    # Cause serve() to terminate soon, allowing the task
    # to finish cleanly when exiting the context manager.
    server.shutdown_soon().

Here open_task_group() refers to some kind of trio-like nursery API. On asyncio this could be a simplified helper, like this:

@asynccontextmanager
async def start_concurrently(async_fn):
    task = asyncio.create_task(async_fn())
    try:
        yield
    finally:
        await task

It could be used like this:

async with start_concurrently(server.serve):
    await server.wait_started()
    # ...
    server.shutdown_soon()

I'm also happy to discuss a threaded equivalent API, for use cases where an event loop isn't readily available or practical (for example, when testing the server using a sync HTTP client).

Then we need to figure out how to make these APIs come to life. :-) Maybe this means we'll need sync/async equivalents, like .wait_started() vs .await_started(), etc.

[Elijas]

That's quite an elaborate pondering, thank you for that! I'd be willing to use all these APIs:)

Just for curiosity

If I understand correctly, having the server.shutdown_soon() implicit in the context itself like in your original example would be considered confusing? At least for my simple use cases, wait_started would immediatelly follow start and context teardown would always implicitly include shutdown_soon anyway - if it's not a part of the lib, I'd write a facade/subclass for it - I think it's simpler to just have something along the lines of

with running_server():
    ... # Make requests to the server, etc.

[florimondmanca]

Well, yes, if we add a .serve_concurrently() method that does the "wait for started on enter" and "shutdown soon then await tasks on exit" thing, that's also an option. Probably also more convenient to use than having to deal with library concurrency APIs. (Let's face it, unlike trio asyncio doesn't offer safe enough APIs that would allow us to tell people to rely on them.)

And then a .run_in_thread() equivalent for threaded contexts.

Now that we know this is the kind of thing we want to have eventually, anyone that'd like to figure out internals and ways to implement this is very much welcome to. :) I'll reopen since I think this is indeed a valuable thing to add.

[roboto84]

For people that stop by here and want to try out florimondmanca's multithreaded approach and are a bit new to uvicorn, Config in the code comes from uvicorn. Configuration below also includes plazmakeks' addition in the case of running into a runtime error described by them.

config = uvicorn.Config(app, host="127.0.0.1", port=5000, log_level="info", loop="asyncio")

Thanks for this discussion all.

[jlaw]

For people that stop by here and want to try out florimondmanca's multithreaded approach and are a bit new to uvicorn, Config in the code comes from uvicorn. Configuration below also includes plazmakeks' addition in the case of running into a runtime error described by them.

config = uvicorn.Config(app, host="127.0.0.1", port=5000, log_level="info", loop="asyncio")

Thanks for this discussion all.

I also encountered this RuntimeError and I did not want to force loop="asyncio", so I implemented this override to run():
freqtrade/freqtrade#4530

I think it would help everyone if the following code was implemented for https://github.com/encode/uvicorn/blob/master/uvicorn/loops/uvloop.py:

  import asyncio
+ import threading

  import uvloop


  def uvloop_setup():
-     asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
+     if (threading.current_thread() is threading.main_thread()):
+         asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
+     else:
+         asyncio.set_event_loop(uvloop.new_event_loop())

[rams3sh]

@florimondmanca I used your code and found the code to be ever running in case self.started was never set. In my case , a port was already in use. But the program does not stop on exception here. Added some output Log below :-

2021-04-08 11:54:41,881 — [Module Name: uvicorn.error] — [PID: 25125] — [Thread : Thread-1] —  INFO — [Method and Line No: serve:64] — Started server process [25125]
INFO:     Waiting for application startup.
2021-04-08 11:54:41,881 — [Module Name: uvicorn.error] — [PID: 25125] — [Thread : Thread-1] —  INFO — [Method and Line No: startup:26] — Waiting for application startup.
INFO:     Application startup complete.
2021-04-08 11:54:41,881 — [Module Name: uvicorn.error] — [PID: 25125] — [Thread : Thread-1] —  INFO — [Method and Line No: startup:38] — Application startup complete.
ERROR:    [Errno 98] error while attempting to bind on address ('127.0.0.1', 8000): address already in use
2021-04-08 11:54:41,881 — [Module Name: uvicorn.error] — [PID: 25125] — [Thread : Thread-1] —  ERROR — [Method and Line No: startup:150] — [Errno 98] error while attempting to bind on address ('127.0.0.1', 8000): address already in use
INFO:     Waiting for application shutdown.
2021-04-08 11:54:41,881 — [Module Name: uvicorn.error] — [PID: 25125] — [Thread : Thread-1] —  INFO — [Method and Line No: shutdown:43] — Waiting for application shutdown.
INFO:     Application shutdown complete.
2021-04-08 11:54:41,882 — [Module Name: uvicorn.error] — [PID: 25125] — [Thread : Thread-1] —  INFO — [Method and Line No: shutdown:46] — Application shutdown complete.
2021-04-08 11:54:42,882 — [Module Name: helper.status ] — [PID: 25125] — [Thread : Thread-2] —  INFO — [Method and Line No: helper:23] — Running ...
2021-04-08 11:54:42,882 — [Module Name: helper.status ] — [PID: 25125] — [Thread : Thread-2] —  INFO — [Method and Line No: helper:23] — Running ...

And this line was never met. Hence I have modified the code as below . Since I am not familiar with the internals of uvicorn server, I may not know if the below code has some edge case or anything that I am missing. Let me know if I am missing anything !!

class Server(uvicorn.Server):

    def install_signal_handlers(self):
        pass

    @contextlib.contextmanager
    def running(self):
        thread = threading.Thread(target=self.run)
        thread.start()
        try:
            while not self.started and thread.is_alive(): # Added a condition for checking if thread is alive 
                time.sleep(1e-3)
            yield
        finally:
            self.should_exit = True
            thread.join()

[Pharisaeus]

I am very confused why this requires some crazy-hoops. I checked the uvicorn.run() code and I don't understand why can't this logic be split into 2 parts:

def run(app, **kwargs):
    runner = create_runner(app, kwargs)
    runner.run()

def create_runner(app, **kwargs):
    config = Config(app, **kwargs)
    server = Server(config=config)

    if (config.reload or config.workers > 1) and not isinstance(app, str):
        logger = logging.getLogger("uvicorn.error")
        logger.warning(
            "You must pass the application as an import string to enable 'reload' or "
            "'workers'."
        )
        sys.exit(1)

    if config.should_reload:
        sock = config.bind_socket()
        supervisor = ChangeReload(config, target=server.run, sockets=[sock])
        return supervisor
    elif config.workers > 1:
        sock = config.bind_socket()
        supervisor = Multiprocess(config, target=server.run, sockets=[sock])
        return supervisor
    else:
        return server

This way someone can still do uvicorn.run() with exactly the same behaviour, but someone else can also manually call runner = uvicorn.create_runner() and then do runner.run(), and have the ability to set runner.should_exit flag.
One comment from me that it would be nice if those 3 types you can get back (Server, ChangeReload and Multiprocess) had some common interface to do this like some close() method. Right now the should_exit flag is either Event or boolean, depending on scenario, and we would have to check the type before doing .set() or = True and this should be delegated to the appropriate class.

[rmorshea]

Anyone care to review #1011? It doesn't solve this issue completely, but it would make the solution a little simpler.

[MauricePasternak]

I have not had success in triggering this to shut down from within a FastAPI route. Could anyone shed light as to how this can be made to work without resorting to psutil process-killing?

import contextlib
import time
import threading
import uvicorn
from fastapi import FastAPI

app = FastAPI()


@app.get("/")
def greet():
    return {"message": "Hello, world"}

@app.get("/kill")
async def kill():
    global server
    server.keep_running = False

class Server(uvicorn.Server):
    def __init__(self, config):
        super().__init__(config)
        self.keep_running = True

    def install_signal_handlers(self):
        pass

    @contextlib.contextmanager
    def run_in_thread(self):
        thread = threading.Thread(target=self.run)
        thread.start()
        try:
            while not self.started:
                time.sleep(1e-3)
            yield
            while self.keep_running:
                time.sleep(1e-3)
        finally:
            self.should_exit = True
            thread.join()

config = uvicorn.Config("main:app", host="127.0.0.1", port=5000, log_level="info")
server = Server(config=config)

if __name__ == "__main__":
    with server.run_in_thread():
        while server.keep_running:
            pass