net/http: SIGKILL, SIGABRT appear to cause server termination yet zombie processes with address still bound persist

[odeke-em]

Please answer these questions before submitting your issue. Thanks!

1. What version of Go are you using (go version)?
   • go version devel +50edddb Tue Aug 2 21:31:58 2016 +0000 darwin/amd64
   • go1.6
2. What operating system and processor architecture are you using (go env)?
   GOARCH="amd64"
   GOBIN="/Users/emmanuelodeke/go/bin"
   GOEXE=""
   GOHOSTARCH="amd64"
   GOHOSTOS="darwin"
   GOOS="darwin"
   GOPATH="/Users/emmanuelodeke/go"
   GORACE=""
   GOROOT="/Users/emmanuelodeke/go/src/go.googlesource.com/go"
   GOTOOLDIR="/Users/emmanuelodeke/go/src/go.googlesource.com/go/pkg/tool/darwin_amd64"
   CC="clang"
   GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/v3/7z434qpx5v3bw0wh8h2myfpw0000gn/T/go-build464848761=/tmp/go-build -gno-record-gcc-switches -fno-common"
   CXX="clang++"
   CGO_ENABLED="1
3. What did you do?
   Ran a simple server. Sent it a SIGKILL, SIGABRT to the program and it exited. However, the socket stays bound to the address of the "killed" server hence any future reuses of the same port/address are impossible.

• Steps, run either https://play.golang.org/p/LorluoHyLW or its variation https://play.golang.org/p/GvjCDDwaNf, get the program's PID and then SIGTERM it.

$ go run server.go --port-addr 4567 &
[1] 62314
2016/08/04 01:44:40 serving at :4567
$ kill -9 62314
[1]+  Killed: 9               go run server.go --port-addr 4567
$ go run server.go --port-addr 4567
2016/08/04 01:46:36 serving at :4567
2016/08/04 01:46:36 listen tcp :4567: bind: address already in use
exit status 1

You can also try with SIGABRT ( kill -6 ) and that produces the same end result in which you can no longer bind to the address when the command is repeated.

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.
https://play.golang.org/p/LorluoHyLW

1. What did you expect to see?
   I expected SIGKILL and SIGABRT to shutdown not only the server and unbind itself from the port just like SIGINT would. If this is acceptable behavior, I haven't found any documentation for it in net/http.
2. What did you see instead?
   When https://play.golang.org/p/LorluoHyLW is rerun, it gives such an error

2016/08/04 01:42:06 listen tcp :4567: bind: address already in use

The issue exists on Go1.6 so this isn't a regression.

[davecheney]

I'm pretty sure this is a generic tcp stack wat. Listening addresses can go
into their version of TIMEWAIT2 until any tcp handshakes in progress have
timed out.

On Thu, 4 Aug 2016, 19:04 Emmanuel T Odeke notifications@github.com wrote:

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

• go version devel +50edddb Tue Aug 2 21:31:58 2016 +0000 darwin/amd64

  • go1.6
    2.

  What operating system and processor architecture are you using (go env
  )?
  GOARCH="amd64"
  GOBIN="/Users/emmanuelodeke/go/bin"
  GOEXE=""
  GOHOSTARCH="amd64"
  GOHOSTOS="darwin"
  GOOS="darwin"
  GOPATH="/Users/emmanuelodeke/go"
  GORACE=""
  GOROOT="/Users/emmanuelodeke/go/src/go.googlesource.com/go"
  GOTOOLDIR="/Users/emmanuelodeke/go/src/
  go.googlesource.com/go/pkg/tool/darwin_amd64"
  CC="clang"
  GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics
  -Qunused-arguments -fmessage-length=0
  -fdebug-prefix-map=/var/folders/v3/7z434qpx5v3bw0wh8h2myfpw0000gn/T/go-build464848761=/tmp/go-build
  -gno-record-gcc-switches -fno-common"
  CXX="clang++"
  CGO_ENABLED="1
  3.

  What did you do?
  Ran a simple server. Sent it a SIGKILL, SIGABRT to the program and it
  exited. However, the socket stays bound to the address of the "killed"
  server hence any future reuses of the same port/address are impossible.

• Steps, run either https://play.golang.org/p/LorluoHyLW or its
  variation https://play.golang.org/p/GvjCDDwaNf, get the program's PID
  and then SIGTERM it.

$ go run server.go --port-addr 4567 &
[1] 62314
2016/08/04 01:44:40 serving at :4567
$ kill -9 62314
[1]+ Killed: 9 go run server.go --port-addr 4567
$ go run server.go --port-addr 4567
2016/08/04 01:46:36 serving at :4567
2016/08/04 01:46:36 listen tcp :4567: bind: address already in useexit status 1

You can also try with SIGABRT ( kill -6 ) and that produces the same end
result in which you can no longer bind to the address when the command is
repeated.

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.
https://play.golang.org/p/LorluoHyLW

What did you expect to see?
I expected SIGKILL and SIGABRT to shutdown not only the server and
unbind itself from the port just like SIGINT would. If this is acceptable
behavior, I haven't found any documentation for it in net/http.
2.

What did you see instead?
When https://play.golang.org/p/LorluoHyLW is rerun, it gives such an
error

2016/08/04 01:42:06 listen tcp :4567: bind: address already in use

The issue exists on Go1.6 so this isn't a regression.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#16599, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAcA03y8E1Oqkv25yId9XJWeNkDZTu5ks5qcasAgaJpZM4JcfL5
.

[rhysh]

The pid you see belongs to the go run command, which compiles your program and runs it as a child process. When you send SIGKILL to that pid, the go run process dies but the child (your program) continues to run.

You'll get the behavior you want if you compile your program once with go build -o ./server server.go and run it as ./server --port-addr 4567 &.

[odeke-em]

Thank you @rhysh for the analysis. Makes sense, and since SIGKILL nor SIGSTOP cannot be stopped, we don't have a way for go run to catch this signal and then kill its children.
For those that need a way to kill these zombies lsof -i :<PORT>

$ lsof -i :4567 | awk '{ print $2 }'
$ # Then find the PID and kill it with -9 e.g
$ lsof -i :4321 | awk '{ print $2 }' | awk 'if ($2 ~ /[0-9 { print $1 }' | while read F;do kill -9 $F;done

Unless @bradfitz or @ianlancetaylor think something else can be done, then this is working as intended.

[bradfitz]

Well, definitely not a net/http bug as the subject implies. But as the net/http owner I will happily push "close". :-)