Graceful shutdown of a tokio-based server

[dominiquelefevre]

Hi,

let us consider a server that spawns a task per each client connection, like the example echo server:

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
    ...
    let mut listener = TcpListener::bind(&addr).await?;
    loop {
        let (mut socket, _) = listener.accept().await?;
        tokio::spawn(async move {handle_one_client(socket)});
    }
}

Suppose that handlers of client requests may use pooled resources like db connections.

Now I want to add graceful shutdown to such server. I want to

1. Stop accepting new connections, and close the listener socket.
2. Signal all client request handlers to clean up and exit. Client request handlers may exit without sending anything to clients, but they have to free any resources that they have acquired: open files, db connections, etc.
3. Cleanup resource pools once I know that no request handler can use those pools.

if I read https://internals.rust-lang.org/t/async-await-the-challenges-besides-syntax-cancellation/10287 correctly, the way to cancel a future is to drop it, which is incompatible with tokio's tokio::spawn(future) which moves future into tokio's runtime. How am I supposed to stop the listener task gracefully?

I can possibly use futures::drop_notify::drop_notify(), and code the listener loop like this:

    let (shutdown, shutdown_notice) = futures::drop_notify::drop_notify();
    pass shutdown to someone who decides when to stop the server;
    loop {
        let r = futures::future::select_all!(vec![listener.accept(), shutdown_notice]).await?;
        if server is shutdown {
            break;
        } else {
            get a socket from r;
            tokio::spawn(async move {handle_one_client(socket)});
        }
    }

But this looks like a lot of boilerplate, and works to cancel a single task only. How to generalise that to canceling many tasks?

P.S. It seems that I am misunderstanding some very basic things about the design of async/await and tokio. That starts with a seemingly contradictive requirements to own a future to cancel it, and to give up the ownership to tokio's runtime to run it. Maybe there is some decent documentation that describes the intended use of async/await with tokio, and typical usage patterns? In particular, I am also interested in propagating context between async tasks to track things like opentracing ids.

[osa1]

Hey @dominiquelefevre,

Gracefully cancelling a future is one of the pain points for me when using Rust's async and tokio. I haven't read a lot of tokio/async code so I don't know how people do this in general, but the pattern I developed is I share a one-shot "stop now" channel or some other channel that can receive a "stop now" message with my tasks, and then use a select! on the main event sources that the task needs and this channel. When the channel returns I simply return; in the task, effectively gracefully terminating it.

Here are a few examples:

• https://github.com/osa1/tiny/blob/cd0cbcad71ede828bd7b8b63cfecff7da3564d19/libtiny_client/src/lib.rs#L602-L650

  This tries to resolve a host address, but also listens to a command channel and stops when receiving a "Quit" message.

• https://github.com/osa1/tiny/blob/cd0cbcad71ede828bd7b8b63cfecff7da3564d19/libtiny_tui/src/lib.rs#L51

  Here I create a channel for signalling "abort" to a task, then pass it to the task. The task (right below that code) uses a select! on the "abort" channel and the main channel it needs to listen to.

It's a painful and verbose process, but that's the only way that I know for gracefully cancelling tasks.

That starts with a seemingly contradictive requirements to own a future to cancel it, and to give up the ownership to tokio's runtime to run it.

My understanding is that tokio owns the tasks, but you can have futures in those tasks that you select! on (e.g. that you poll occasionally), and when you return from your taks those futures are dropped and cancelled.

It's possible that I'm misunderstanding this though.

[carllerche]

Relates to #1879.

[carllerche]

Basically, graceful server shutdown is something we need to figure out how we want to handle API wise.

We could add a cancel function on JoinHandle, but I hesitate to do that as the only way to implement would be to force drop the task and that would be abrupt and not graceful.

The linked issue above describes some potential strategies to model structured spawn where the child task is spawned w/ a cancellation handle that it can use to gracefully shutdown.

[vojtechkral]

Hi. Just a heads-up: with easier task cancelation in mind, I recently contributed take_until() to futures. With Notify in tokio and take_until() in futures, cancellable tasks are somewhat easier now, although you still do need some boilerplate code to distribute the cancelation Notify...

[Finomnis]

I attempted to provide a solution for the distribution of a cancellation notify:
https://crates.io/crates/tokio-graceful-shutdown

[carllerche]

I have converted this to a Discussion.

Also worth noting that the Mini-Redis example shows how to do graceful shutdown: https://github.com/tokio-rs/mini-redis#graceful-shutdown