Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net: poor performance on tcp accept / close workload #32592

Closed
Anteoy opened this issue Jun 13, 2019 · 6 comments
Closed

net: poor performance on tcp accept / close workload #32592

Anteoy opened this issue Jun 13, 2019 · 6 comments

Comments

@Anteoy
Copy link

@Anteoy Anteoy commented Jun 13, 2019

What version of Go are you using (go version)?

$ go version
go version go1.10.8 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN="/home/zhoudazhuang/gobin/"
GOCACHE="/home/zhoudazhuang/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/zhoudazhuang/db11/jm/pro"
GORACE=""
GOROOT="/home/zhoudazhuang/usr/local/go1.10.8/go"
GOTMPDIR=""
GOTOOLDIR="/home/zhoudazhuang/usr/local/go1.10.8/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build040701016=/tmp/go-build -gno-record-gcc-switches"

What did you do?

tl, err := net.ListenTCP("tcp4", tcpAddr)
for {
		s := time.Now().UnixNano()
		conn, err := tl.AcceptTCP()
		e := time.Now().UnixNano()
		fmt.Printf("cost time: %s, %d \n",time.Now(), e-s)
		if err != nil {
			fmt.Printf("get err %s\n",err)
		}
		if conn != nil {
			go conn.Close()
		}
}

my linux is :

Linux ll-025048236-FWWG.AppPZFW.prod.bj1 2.6.32-642.el6.x86_64 #1 SMP Tue May 10 17:27:01 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
cat /proc/sys/net/ipv4/tcp_max_syn_backlog && cat /proc/sys/net/core/somaxconn
65535
65535
128G 30C

online env, I have probabily more than 100k tcp connection / each second, I have lots of close-wait conn. When I accept a conn, I do nothing,only call conn.Close then get Next conn by for. But my close-wait tcp conn more and more ,even more than few million close-wait.
I statistic by:

ss -s && ss -an | grep CLOSE-WAIT | grep 8082 | wc -l

or:

ss -s && ss -anp state close-wait 'sport = :8082' | wc -l

What did you expect to see?

I suspect it cause by conn.Close()
Is this the standard performance or something I can do?
I want to see there is no or few of close-wait tcp , or my linux will crash by memory .

What did you see instead?

even more than few million close-wait.

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jun 13, 2019

If you accept a lot of TCP connections and then immediately close them, it seems normal to accumulate sockets in CLOSE_WAIT status. That's how TCP works.

You may be able to reduce the number of such sockets by calling conn.SetLinger(0).

@Anteoy

This comment has been minimized.

Copy link
Author

@Anteoy Anteoy commented Jun 13, 2019

I see about the conn.SetLinger(0)

// If sec == 0, the operating system discards any unsent or
// unacknowledged data.

The unacknowledged is data trans ack? or like FIN or SYN tcp package ack?
Is there any side effect ?
Is it effect the client ?

@ALTree ALTree changed the title net tcp accept close, performance is too poor net: poor performance on tcp accept / close workload Jun 13, 2019
@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jun 13, 2019

Is there anything for us to do with this issue?

@Anteoy

This comment has been minimized.

Copy link
Author

@Anteoy Anteoy commented Jun 14, 2019

I want to observe for a while

@Anteoy

This comment has been minimized.

Copy link
Author

@Anteoy Anteoy commented Jun 17, 2019

THX

@Anteoy Anteoy closed this Jun 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.