Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTTP graceful Shutdown Fail on SSE / event-stream Connections #41344

Closed
Indribell opened this issue Sep 11, 2020 · 2 comments
Closed

HTTP graceful Shutdown Fail on SSE / event-stream Connections #41344

Indribell opened this issue Sep 11, 2020 · 2 comments

Comments

@Indribell
Copy link

@Indribell Indribell commented Sep 11, 2020

What version of Go are you using (go version)?

go version go1.15.2 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/vscode/.cache/go-build"
GOENV="/home/vscode/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="0"
GOMOD="/data/indribell/main2/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build967309556=/tmp/go-build -gno-record-gcc-switches"

What did you do?

// Start the server and set things up for a graceful shutdown of the server
go func() {
	if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
		log.Fatalf("listen: %s\n", err)
	}
}()
log.Println("SSE Server Started")

<-done
log.Println("SSE Server Stopping")

// Create a new context with a timeout duration. It helps allowing
// timeout duration to all active connections in order for them to
// finish their job. Any connections that wont complete within the
// allowed timeout duration gets halted.
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()

if err := srv.Shutdown(ctx); err != nil {
	log.Fatalf("SSE Server Shutdown Failed:%+v", err)
}
log.Println("SSE Server Exited Properly")

What did you expect to see?

SSE Server Exited Properly

What did you see instead?

SSE Server Shutdown Failed:context deadline exceeded

The issue is that any connection opened with a:

response.Header().Set("Content-Type", "text/event-stream")

Keep persisting even when a graceful shutdown is initiated. There needs to be a check if no data is being transmitted over the stream and then it needs to be cut.

When no connection is (yet) established, graceful shutdowns work perfectly and fast ( withing 1 second ).

@seankhliao
Copy link
Contributor

@seankhliao seankhliao commented Sep 11, 2020

Shutdown works by first closing all open listeners, then closing all idle connections, and then waiting indefinitely for connections to return to idle and then shut down.

you need to return from your handlers first for the connections to be considered idle (not used for active HTTP requests)

@davecheney
Copy link
Contributor

@davecheney davecheney commented Sep 12, 2020

This appears to be working as intended so I am closing this issue as answered. Please comment if you disagree.

Unlike many projects, the Go project does not use GitHub Issues for general discussion or asking questions. GitHub Issues are used for tracking bugs and proposals only.

For asking questions, see:

@davecheney davecheney closed this Sep 12, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.