Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: File descriptor leakage #20413

Closed
magisterquis opened this issue May 18, 2017 · 2 comments

Comments

Projects
None yet
3 participants
@magisterquis
Copy link

commented May 18, 2017

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.8.1 openbsd/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="openbsd"
GOOS="openbsd"
GOPATH="/home/stuart/go"
GORACE=""
GOROOT="/home/stuart/.go/1.8.1"
GOTOOLDIR="/home/stuart/.go/1.8.1/pkg/tool/openbsd_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -fmessage-length=0"
CXX="g++"
CGO_ENABLED="0"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

What did you do?

Build Host:

GOOS=linux go build -o f.linux.amd64 f.go

Execution Host:

./f.linux.amd64

Code: http://play.golang.org/p/gKiRi7x6V5

What did you expect to see?

HTTP requests, and file descriptors deallocated.

What did you see instead?

File descriptors are not released. lsof -Pnp <pid> indicates sockets hang around once they're no longer in a CLOSE_WAIT state.

$ lsof -Pnp 19380
COMMAND     PID            USER   FD      TYPE   DEVICE SIZE/OFF     NODE NAME
f.linux.a 19380 stuart  cwd       DIR    253,3     4096 75497664 /home/stuart.mcmurray
f.linux.a 19380 stuart  rtd       DIR    253,3     4096      128 /
f.linux.a 19380 stuart  txt       REG    253,3  5523864 75509093 /home/stuart.mcmurray/f.linux.amd64
f.linux.a 19380 stuart    0u      CHR    136,0      0t0        3 /dev/pts/0
f.linux.a 19380 stuart    1u      CHR    136,0      0t0        3 /dev/pts/0
f.linux.a 19380 stuart    2u      CHR    136,0      0t0        3 /dev/pts/0
f.linux.a 19380 stuart    3u     sock      0,6      0t0 48758872 protocol: TCP
f.linux.a 19380 stuart    4r      CHR      1,9      0t0     1033 /dev/urandom
f.linux.a 19380 stuart    5u  a_inode      0,9        0     5719 [eventpoll]
f.linux.a 19380 stuart    6u     sock      0,6      0t0 48753565 protocol: TCP
f.linux.a 19380 stuart    7u     sock      0,6      0t0 48753578 protocol: TCP
f.linux.a 19380 stuart    8u     sock      0,6      0t0 48757923 protocol: TCP
f.linux.a 19380 stuart    9u     sock      0,6      0t0 48757941 protocol: TCP
f.linux.a 19380 stuart   10u     sock      0,6      0t0 48757954 protocol: TCP
f.linux.a 19380 stuart   11u     IPv4 48757964      0t0      TCP 10.40.15.176:49254->52.23.67.15:443 (CLOSE_WAIT)
f.linux.a 19380 stuart   12u     sock      0,6      0t0 48758954 protocol: TCP
f.linux.a 19380 stuart   13u     sock      0,6      0t0 48758964 protocol: TCP
f.linux.a 19380 stuart   14u     sock      0,6      0t0 48757994 protocol: TCP
f.linux.a 19380 stuart   15u     sock      0,6      0t0 48759828 protocol: TCP
f.linux.a 19380 stuart   16u     IPv4 48756119      0t0      TCP 10.40.15.176:49259->52.23.67.15:443 (CLOSE_WAIT)
f.linux.a 19380 stuart   17u     IPv4 48756125      0t0      TCP 10.40.15.176:52080->34.197.186.182:443 (CLOSE_WAIT)
f.linux.a 19380 stuart   18u     IPv4 48758042      0t0      TCP 10.40.15.176:49261->52.23.67.15:443 (CLOSE_WAIT)
f.linux.a 19380 stuart   19u     IPv4 48759033      0t0      TCP 10.40.15.176:38018->107.23.175.12:443 (CLOSE_WAIT)
f.linux.a 19380 stuart   20u     IPv4 48758057      0t0      TCP 10.40.15.176:37774->34.192.54.247:443 (CLOSE_WAIT)
f.linux.a 19380 stuart   21u     IPv4 48756183      0t0      TCP 10.40.15.176:38020->107.23.175.12:443 (CLOSE_WAIT)
f.linux.a 19380 stuart   22u     IPv4 48759062      0t0      TCP 10.40.15.176:49265->52.23.67.15:443 (CLOSE_WAIT)
f.linux.a 19380 stuart   23u     IPv4 48758075      0t0      TCP 10.40.15.176:37777->34.192.54.247:443 (CLOSE_WAIT)
f.linux.a 19380 stuart   24u     IPv4 48759073      0t0      TCP 10.40.15.176:37778->34.192.54.247:443 (CLOSE_WAIT)
f.linux.a 19380 stuart   25u     IPv4 48758086      0t0      TCP 10.40.15.176:38024->107.23.175.12:443 (CLOSE_WAIT)
f.linux.a 19380 stuart   26u     IPv4 48759088      0t0      TCP 10.40.15.176:52089->34.197.186.182:443 (CLOSE_WAIT)
f.linux.a 19380 stuart   27u     IPv4 48758103      0t0      TCP 10.40.15.176:38026->107.23.175.12:443 (CLOSE_WAIT)
-snip-

It seems net/http doesn't close file descriptors when it's done with them. The fd ulimit is set to 1024.

@bradfitz

This comment has been minimized.

Copy link
Member

commented May 18, 2017

You don't close the Body in this case:

        if http.StatusOK != res.StatusCode {
                log.Printf("Ping Status: %v", res.Status)
                return
        }

I can't reproduce this on a server that's only returning 200 OKs

After a couple minutes:

$ lsof -p 15475
COMMAND   PID     USER   FD   TYPE DEVICE SIZE/OFF   NODE NAME
repro   15475 bradfitz  cwd    DIR    8,1    16384 531000 /home/bradfitz
repro   15475 bradfitz  rtd    DIR    8,1     4096      2 /
repro   15475 bradfitz  txt    REG    8,1  4057184 655685 /tmp/go-build186024032/command-line-arguments/_obj/exe/repro
repro   15475 bradfitz  mem    REG    8,1  1738176 540080 /lib/x86_64-linux-gnu/libc-2.19.so
repro   15475 bradfitz  mem    REG    8,1   137384 532910 /lib/x86_64-linux-gnu/libpthread-2.19.so
repro   15475 bradfitz  mem    REG    8,1   140928 533432 /lib/x86_64-linux-gnu/ld-2.19.so
repro   15475 bradfitz    0u   CHR  136,2      0t0      5 /dev/pts/2
repro   15475 bradfitz    1u   CHR  136,2      0t0      5 /dev/pts/2
repro   15475 bradfitz    2u   CHR  136,2      0t0      5 /dev/pts/2
repro   15475 bradfitz    4u  0000    0,9        0   7634 anon_inode
repro   15475 bradfitz    5r   CHR    1,9      0t0   1033 /dev/urandom

This is well tested in the net/http package. Be sure to always close your response Body.

Closing, but let me know if I'm missing something.

@bradfitz bradfitz closed this May 18, 2017

@magisterquis

This comment has been minimized.

Copy link
Author

commented May 18, 2017

That was it. My mistake.

I appreciate it.

@mikioh mikioh changed the title File descriptor leakage net/http: File descriptor leakage Jun 2, 2017

@golang golang locked and limited conversation to collaborators Jun 2, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.