proposal: net/http: context based graceful server shutdown

[marwan-at-work]

Proposal

I propose adding two new methods to the net/http *Server struct: ListenAndServeContext and ListenAndServeTLSContext that will handle gracefully shutting down the server upon a context cancellation signal.

To accompany these two methods, we need a graceful shutdown timeout which can either live as a field in the Server struct, or an argument to the new methods.

The signature can either be:

// ListenAndServeContext is like ListenAndServer but will call s.Shutdown() when the given
// context sends a cancellation signal. If the ShutdownTimeout field is set, it will be used
// as timeout duration for the Shutdown method.
func (s *Server) ListenAndServeContext(ctx context.Context) error
func (s *Server) ListenAndServeTLSContext(ctx context.Context, certFile, keyFile string) error

type Server struct {
  ...
  ShutdownTimeout time.Duration
  ...
}

OR

func (s *Server) ListenAndServeContext(ctx context.Context, timeout time.Duration) error
func (s *Server) ListenAndServeTLSContext(ctx context.Context, timeout time.Duration, certFile, keyFile string) error

It might also be worth noting that there are two more methods (Serve, and ServeTLS) that can arguably take a context as well. I rarely ever encounter explicit net.Listeners being passed in when starting an http server so I'll leave that up for discussion.

The examples below will assume option 2 above as it makes it more explicit for the user to pass a Shutdown timeout instead of it being tucked away as a struct field.

Code Examples

The code from the caller's perspective will look something like this:

ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt)
defer cancel()

server := &http.Server{
  Addr: addr,
  Handler: handler,
}

if err := server.ListenAndServeContext(ctx, time.Second); err != nil {
  log.Fatal(err)
}

Implementation

The underlying implementation would abstract the subtle details to handle graceful shutdowns:

func (s *Server) ListenAndServe(ctx context.Context, shutdownTimeout time.Duration) error {
  return s.ListenAndServeTLS(ctx, shutdownTimeout, "", "")
}

func (s *Server) ListenAndServeTLS(ctx context.Context, shutdownTimeout time.Time, certFile, keyFile string) error {
	serverErr := make(chan error, 1)
	go func() {
		// Capture ListenAndServe errors such as "port already in use".
		// However, when a server is gracefully shutdown, it is safe to ignore errors
		// returned from this method (given the select logic below), because
		// Shutdown causes ListenAndServe to always return http.ErrServerClosed.
		if certFile != "" && keyFile != "" {
			serverErr <- s.ListenAndServeTLS(certFile, keyFile)
		} else {
			serverErr <- s.ListenAndServe()
		}
	}()
	var err error
	select {
	case <-ctx.Done():
		ctx, cancel := context.WithTimeout(context.Background(), shutdownTimeout)
		defer cancel()
		err = s.Shutdown(ctx)
	case err = <-serverErr:
	}
	return err
}

Notably, the implementation above would return a nil error when a server is gracefully and successfully shutdown without hitting the timeout. We could also explicitly return the ctx.Error() or a new error variable if we never want to return a nil error. But I think a nil error is a fine signal for saying a server was successfully terminated.

Finally, there's a question of what a 0 duration would mean:

1. Either it means no graceful shutdown since the context being passed to s.Shutdown will immediately be cancelled.
2. Or it means wait forever such as the equivalent of s.Shutdown(context.Background().

The first option seems more rational though the second might be nice for convenience but it would have to be explicitly documented.

Rationale

Many people today run Go servers while not implementing graceful shutdown in the first place (for example calling the global http.ListenAndServe function) or implementing it but missing edge cases such as the following:

1. Misusing shutdown.

go func() {
  if err := srv.ListenAndServer(); err != nil {
    log.Fatal(err) // will immediately log http.ErrServerClosed and exit.
  }
}()

4. Ignoring legitimate ListenAndServe errors:

go func() { srv.ListenAndServe() }()

And probably other subtleties. I have myself made those mistakes many items and ended up abstracting the logic into its own context: https://github.com/marwan-at-work/serverctx/blob/main/serverctx.go

Looking at private code within my company I also noticed each team does a flavor of the above which seems like a good case for code re-use.

Furthermore, looking at the standard library we already have similar patterns such as exec.Command and exec.CommandContext which could make this a natural fit for Go developers already familiar with these APIs.

[ianlancetaylor]

CC @neild @bradfitz

[neild]

This proposal passes a Context and a graceful shutdown timeout to Server.ListenAndServeContext. The Context bounds the time until the server starts graceful shutdown, and the timeout bounds how long the graceful shutdown period lasts.

The existing Server.Shutdown method takes a Context to bound the graceful shutdown period.

Under this proposal, therefore, we'd have one API that takes a Context to bound the server lifetime but not the shutdown period, and a different and incompatible API which takes a Context to bound the graceful shutdown period but not the server lifetime. That seems confusing. It also means the ListenAndServeContext function would only be usable in cases where there is no graceful shutdown period or the grace period is strictly time-bounded.

I think this proposal is addressing two issues at the same time:

• You can't bound a http.Server's lifetime using a Context without spinning up a goroutine and manually watching for context cancellation. I don't know how much of an issue this is in practice; how often do people want to stop a server using a Context rather than by calling Server.Shutdown?
• The interactions between Server.Shutdown and Server.ListenAndServe are somewhat sharp-edged: ListenAndServe returns immediately with an error, when returning nil after shutdown has completed might be friendlier in most cases.

Perhaps it might be simpler to address the second point without introducing a context to ListenAndServe.

[natefinch]

how often do people want to stop a server using a Context rather than by calling Server.Shutdown?

Basically always? Context is the canonical way to control the lifetime of a call. This is why signal.NotifyContext was added to the standard library, because it was so common to want to hook these two things up.

I wonder if the solution is to add graceful shutdown semantics right into context.Context, and then you'd only need ListenAndServeCtx, and the context would fully contain graceful exit semantics as well as simple cancel semantics.

And this would also then give everyone who uses contexts a one-stop shop for graceful shutdowns.

I think we steal the idea from Shutdown(ctx) - implement a separate context that only controls the lifetime of Shutdown logic. It'll be a context within a context, but if we can word things well, I think it'll be ok.

// ContextForShutdown returns the Context that controls the lifetime of the logic that 
// runs when ctx's Done channel is closed.
func ContextForShutdown(ctx Context) Context

// WithShutdownTimeout returns a context that puts a timeout on the context returned by
// ContextForShutdown, and a CancelFunc that closes the done channel on the shutdown context. 
func WithShutdownTimeout(parent Context, timeout time.Duration) (ctx Context, cancelShutdown CancelFunc)

// WithShutdownCancel returns a context and a CancelFunc that will close the done
// channel on the context returned by ContextForShutdown for ctx. 
func WithShutdownCancel(parent Context) (ctx Context, cancelShutdown CancelFunc)

I use Shutdown here to try to avoid the confusion of reusing "Cancel". Shutdown being the logic that runs after the original context is marked done.

// in user code, this is all we'd need for graceful shutdown of an application
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt)
defer stop() 
ctx, cancelShutdown := context.WithShutdownTimeout(ctx, timeout)
defer cancelShutdown()
return server.ListenAndServeCtx(ctx)

And then once the original context gets closed by the Signal, http.Server would call context.ContextForShutdown on the context it was passed by ListenAndServeCtx, and use that to pass into Shutdown.

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

[earthboundkid]

I'm agnostic about the right API here, but I do think this is a common need and a bit of a PITA to code correctly.

Maybe it accepts a callback to run when ctx1 is cancelled and that callback returns the context for the shutdown?

[DeedleFake]

In the event of a nil Server.BaseContext, would the provided Context be used?

[rsc]

It sounds like having some kind of graceful shutdown is a common need that might be worth filling. People seem less enthusiastic about these specific API details. Does anyone want to propose a simpler way?

[marwan-at-work]

I'm all ears (or eyes I guess) for proposing a simpler way 👍🏼

One alternative we can do is have server.Server take a context as an optional field (in addition to context timeout) and this way *Server.ListenAndServe (and friends) can check if the context field is not nil and it would launch a goroutine under the hood that manages the graceful shutdown. This would mean no new additional methods but only two additional fields.

Something like this:

type Server struct {
  ...
  ShutdownTimeout time.Duration
  Context context.Context
  ...
}

and you would just call server.ListenAndServer() as you normally do. People managing ListenAndServe manually won't be broken because the Context will be nil. And if people decide to add a Context, then they are explicitly choosing the new API and would not need to manage graceful shutdown anymore.

Personally I still prefer the Go convention of explicitly passing context as a method/function argument but there's precedent here with net/http.Request storing the context as a field. So I'm happy either way

[seankhliao]

I think having http.Server itself support context based shutdowns isn't enough, often there are other background tasks, connection upgrades, or even multiple http.Servers running which all need to be coordinated and accounted for in a graceful shutdown.

I would rather see something like a version of github.com/oklog/run that can support both context-based and call-based cancellation, allowing existing code to plug in to coordinated shutdowns.

[neild]

I think that any proposal here needs to start with a clear goal.

The original post here includes a function which shows how to run an http.Server with a context-bounded lifetime. Lightly modified:

func ListenAndServeTLS(ctx context.Context, server *http.Server, shutdownTimeout time.Duration, certFile, keyFile string) error {
	serverErr := make(chan error, 1)
	go func() {
		serverErr <- server.ListenAndServeTLS(certFile, keyFile)
	}()
	select {
	case <-ctx.Done():
		ctx, cancel := context.WithTimeout(context.Background(), shutdownTimeout)
		defer cancel()
		err = s.Shutdown(ctx)
	case err := <-serverErr:
		return err
	}
}

That doesn't look too bad to me.

The least elegant part to my eyes is the handling of shutdownTimeout, which creates a new background context to apply the graceful shutdown timeout. However, this lack of elegance is, I think, implicit in using contexts to bound the lifetime of operations: We cannot both bound the server lifetime with a context and have a graceful shutdown, because graceful shutdown requires extending the server lifetime past the point the server context expires.

This argues to me that a context is not the right way to bound the lifetime of an http.Server, and that the API we have today with Server.Close and Server.Shutdown explicitly terminating a server's lifetime is the better approach.

[earthboundkid]

There are already ListenAndServe, ListenAndServeTLS, Serve, ServeTLS. Adding more context variant methods seems bad, so adding struct fields seems like lesser evil. The approach of using callbacks is pretty flexible. Maybe BeforeFunc func(*http.Server) error (not sure if this is a good name).

server := &http.Server{
  Addr: addr,
  Handler: handler,
  BeforeFunc: func(s *http.Server) {
        log.Print("starting up")
	c := make(chan os.Signal, 1)
	signal.Notify(c, os.Interrupt)
	<- c
        log.Print("shutting down")
	ctx, cancel := context.WithTimeout(context.Background(), shutdownTimeout)
	defer cancel()
	return s.Shutdown(ctx)
  },
}

if err := server.ListenAndServeContext(ctx, time.Second); err != nil {
  log.Fatal(err)
}

[bcmills]

This argues to me that a context is not the right way to bound the lifetime of an http.Server, and that the API we have today with Server.Close and Server.Shutdown explicitly terminating a server's lifetime is the better approach.

I concur. My mental model of context.Context is that cancellation means “the caller is no longer interested in the results of this call”, and through that lens passing a Context to ListenAndServeContext would mean “the caller is no longer interested in the output of the server”. That sounds to me like a hard shutdown, not a graceful one.