socket leak on server side

[hicqu]

Bug Report

What version of TiKV are you using?

3.0.1, 3.0.3

What operating system and CPU are you using?

Linux.

Steps to reproduce

1. Start PD and TiKV
2. Write a client with go or rust,call batch_commands to send a request, and then close the connection (for go) or drop the client (for rust).
3. tcpdump shows that

17:19:59.193810 IP localhost.43474 > localhost.60003: Flags [S], seq 3116894430, win 43690, options [mss 65495,sackOK,TS val 4083567442 ecr 0,nop,wscale 7], length 0
17:19:59.193835 IP localhost.60003 > localhost.43474: Flags [S.], seq 1484174627, ack 3116894431, win 43690, options [mss 65495,sackOK,TS val 4083567442 ecr 4083567442,nop,wscale 7], length 0
17:19:59.193847 IP localhost.43474 > localhost.60003: Flags [.], ack 1, win 342, options [nop,nop,TS val 4083567442 ecr 4083567442], length 0
17:19:59.194042 IP localhost.43474 > localhost.60003: Flags [P.], seq 1:391, ack 1, win 342, options [nop,nop,TS val 4083567442 ecr 4083567442], length 390
17:19:59.194050 IP localhost.60003 > localhost.43474: Flags [.], ack 391, win 350, options [nop,nop,TS val 4083567442 ecr 4083567442], length 0
17:19:59.194638 IP localhost.60003 > localhost.43474: Flags [P.], seq 1:70, ack 391, win 350, options [nop,nop,TS val 4083567443 ecr 4083567442], length 69
17:19:59.194651 IP localhost.43474 > localhost.60003: Flags [.], ack 70, win 342, options [nop,nop,TS val 4083567443 ecr 4083567443], length 0
17:19:59.194703 IP localhost.43474 > localhost.60003: Flags [P.], seq 391:417, ack 70, win 342, options [nop,nop,TS val 4083567443 ecr 4083567443], length 26
17:19:59.194772 IP localhost.60003 > localhost.43474: Flags [P.], seq 70:109, ack 417, win 350, options [nop,nop,TS val 4083567443 ecr 4083567443], length 39
17:19:59.234333 IP localhost.43474 > localhost.60003: Flags [.], ack 109, win 342, options [nop,nop,TS val 4083567483 ecr 4083567443], length 0
17:20:02.197605 IP localhost.43474 > localhost.60003: Flags [F.], seq 417, ack 109, win 342, options [nop,nop,TS val 4083570446 ecr 4083567443], length 0
17:20:02.197723 IP localhost.60003 > localhost.43474: Flags [P.], seq 109:219, ack 418, win 350, options [nop,nop,TS val 4083570446 ecr 4083570446], length 110
17:20:02.197747 IP localhost.43474 > localhost.60003: Flags [R], seq 3116894848, win 0, length 0

4. lsof shows

$ sudo lsof -p 60874|grep -vE "FIFO|a_inode|DIR|REG|CHR"
COMMAND     PID   USER   FD      TYPE    DEVICE  SIZE/OFF      NODE NAME
tikv-serv 60874 qupeng    7u     IPv4 259497323       0t0       TCP localhost:34422->localhost:60001 (ESTABLISHED)
tikv-serv 60874 qupeng   77u     IPv4 259417052       0t0       TCP localhost:60003 (LISTEN)
tikv-serv 60874 qupeng  131u     IPv4 259497324       0t0       TCP localhost:60003 (LISTEN)
tikv-serv 60874 qupeng  212u     IPv4 259497325       0t0       TCP localhost:60003 (LISTEN)
tikv-serv 60874 qupeng  233u     IPv4 259521582       0t0       TCP localhost:60013 (LISTEN)
tikv-serv 60874 qupeng  234u     IPv4 259497326       0t0       TCP localhost:60003 (LISTEN)
tikv-serv 60874 qupeng  410u     sock       0,7       0t0 259427948 protocol: TCP
tikv-serv 60874 qupeng  413u     sock       0,7       0t0 259494646 protocol: TCP
tikv-serv 60874 qupeng  416u     sock       0,7       0t0 259273887 protocol: TCP
tikv-serv 60874 qupeng  419u     sock       0,7       0t0 259528650 protocol: TCP
tikv-serv 60874 qupeng  422u     sock       0,7       0t0 259537857 protocol: TCP
tikv-serv 60874 qupeng  423u     sock       0,7       0t0 259560575 protocol: TCP

It proves that tikv server doesn't call close on the fd correctly.

What did you expect?

TiKV should call close on sockets correctly.

What did happened?

TiKV doesn't call close on sockets correctly.