数据并发读写冲突 #277

buexplain · 2023-03-15T05:59:54Z

第一个是：

==================
WARNING: DATA RACE
Read at 0x00c006aa4040 by goroutine 1993:
  github.com/lesismal/nbio.(*poller).deleteConn()
      F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:89 +0xdb
  github.com/lesismal/nbio.(*Conn).Close()
      F:/netsvr/vendor/github.com/lesismal/nbio/conn_std.go:275 +0x1c7
  github.com/lesismal/nbio/nbhttp/websocket.(*WebsocketReader).handleWsMessage()
      F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/websocket/upgrader.go:663 +0x459     
  github.com/lesismal/nbio/nbhttp/websocket.(*WebsocketReader).handleProtocolMessage.func1()
      F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/websocket/upgrader.go:620 +0x89      
  github.com/lesismal/nbio.(*Conn).Execute.func1.1()
      F:/netsvr/vendor/github.com/lesismal/nbio/conn.go:127 +0x55
  github.com/lesismal/nbio.(*Conn).Execute.func1()
      F:/netsvr/vendor/github.com/lesismal/nbio/conn.go:128 +0x58
  github.com/lesismal/nbio/taskpool.(*MixedPool).callWitoutRecover()
      F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/mixedpool.go:38 +0x72
  github.com/lesismal/nbio/taskpool.(*MixedPool).callWitoutRecover-fm()
      <autogenerated>:1 +0x44
  github.com/lesismal/nbio/taskpool.(*MixedPool).Go.func1()
      F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/mixedpool.go:45 +0x77

Previous write at 0x00c006aa4040 by goroutine 1523:
  github.com/lesismal/nbio.(*Conn).readTCP()
      F:/netsvr/vendor/github.com/lesismal/nbio/conn_std.go:110 +0x10b
  github.com/lesismal/nbio.(*Conn).read()
      F:/netsvr/vendor/github.com/lesismal/nbio/conn_std.go:95 +0x7a
  github.com/lesismal/nbio.(*poller).readConn()
      F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:57 +0x8d
  github.com/lesismal/nbio.(*poller).addConn.func1()
      F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:77 +0x47

Goroutine 1993 (running) created at:
  github.com/lesismal/nbio/taskpool.(*MixedPool).Go()
      F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/mixedpool.go:44 +0x15e
  github.com/lesismal/nbio/taskpool.(*MixedPool).Go-fm()
      <autogenerated>:1 +0x44
  github.com/lesismal/nbio.(*Conn).Execute()
      F:/netsvr/vendor/github.com/lesismal/nbio/conn.go:115 +0x341
  github.com/lesismal/nbio.(*Conn).Execute-fm()
      <autogenerated>:1 +0x44
  github.com/lesismal/nbio/nbhttp/websocket.(*WebsocketReader).handleProtocolMessage()
      F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/websocket/upgrader.go:619 +0x2a8
  github.com/lesismal/nbio/nbhttp/websocket.(*WebsocketReader).Read()
      F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/websocket/upgrader.go:540 +0x12f9
  github.com/lesismal/nbio/nbhttp.(*Parser).Read()
      F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/parser.go:153 +0x4c1
  github.com/lesismal/nbio/nbhttp.(*Engine).DataHandler()
      F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:514 +0xce
  github.com/lesismal/nbio/nbhttp.(*Engine).DataHandler-fm()
      <autogenerated>:1 +0x6d
  github.com/lesismal/nbio/nbhttp.NewEngine.func6()
      F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:945 +0x8e
  github.com/lesismal/nbio.(*Conn).readTCP()
      F:/netsvr/vendor/github.com/lesismal/nbio/conn_std.go:122 +0x352
  github.com/lesismal/nbio.(*Conn).read()
      F:/netsvr/vendor/github.com/lesismal/nbio/conn_std.go:95 +0x7a
  github.com/lesismal/nbio.(*poller).readConn()
      F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:57 +0x8d
  github.com/lesismal/nbio.(*poller).addConn.func1()
      F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:77 +0x47

Goroutine 1523 (finished) created at:
  github.com/lesismal/nbio.(*poller).addConn()
      F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:77 +0x204
  github.com/lesismal/nbio.(*Engine).AddConn()
      F:/netsvr/vendor/github.com/lesismal/nbio/engine.go:211 +0x10d
  github.com/lesismal/nbio/nbhttp.(*Engine).AddConnNonTLSNonBlocking()
      F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:590 +0x5ad
  github.com/lesismal/nbio/nbhttp.(*Engine).AddConnNonTLSNonBlocking-fm()
      <autogenerated>:1 +0x6d
  github.com/lesismal/nbio/nbhttp.(*Engine).listen.func1()
      F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:273 +0x250
==================

第二个是：
https://github.com/lesismal/nbio/blob/master/nbhttp/engine.go#L426
读取map长度的时候没有加锁

lesismal · 2023-03-15T06:45:14Z

感谢关注和反馈！

已知的这些数组、map的读取是并发安全的，如果为了避免race warnign而加锁会影响性能。
之前有些commit尝试使用//go:norace消除race warnign，但实框架实现中涉及很多这种并发race、要全面消除race warnign会做太多不必要的修改、所以又去掉了。
具体的告警OP可以看下实际代码或实际运行中是否会造成数据错乱或者其他bug，如果确实有问题会进行修复，否则请忽略race warning。

以前也有一些相关issue，有兴趣也可以issue里搜下race相关。

buexplain · 2023-03-15T09:06:15Z

如果只是竞争告警，倒是无所谓。我又观察了一波，发现疑似协程泄漏了。情况如下：
我服务端启动后，记录下协程堆栈信息，然后并发的发起websocket连接到服务端，当连接达到500多个的时候，客户端并发的关闭连接，客户端进程关闭后，服务端的协程堆栈信息比服务端刚启动的时候多了一条协程。
我感觉它(goroutine 6173 [IO wait])是泄漏了，还请大佬检查一下。

服务端启动后，没有任何连接进来时的协程堆栈

goroutine 59 [running]:
runtime/pprof.writeGoroutineStacks({0xd7a240, 0xc0001b7500})
	D:/Go/src/runtime/pprof/pprof.go:703 +0x85
runtime/pprof.writeGoroutine({0xd7a240, 0xc0001b7500}, 0x2)
	D:/Go/src/runtime/pprof/pprof.go:692 +0x4b
runtime/pprof.(*Profile).WriteTo(0x10e0f60, {0xd7a240, 0xc0001b7500}, 0xc?)
	D:/Go/src/runtime/pprof/pprof.go:329 +0x1bb
net/http/pprof.handler.ServeHTTP({0xc00001c911, 0x9}, {0xd7d660, 0xc0001b7500}, 0x0?)
	D:/Go/src/net/http/pprof/pprof.go:259 +0x57f
net/http/pprof.Index({0xd7d660?, 0xc0001b7500}, 0xc000182b00)
	D:/Go/src/net/http/pprof/pprof.go:376 +0x12c
net/http.HandlerFunc.ServeHTTP(0xcbbfb8, {0xd7d660, 0xc0001b7500}, 0x0?)
	D:/Go/src/net/http/server.go:2122 +0x4e
net/http.(*ServeMux).ServeHTTP(0xc00001c91b?, {0xd7d660, 0xc0001b7500}, 0xc000182b00)
	D:/Go/src/net/http/server.go:2500 +0xc6
net/http.serverHandler.ServeHTTP({0xc0000314a0?}, {0xd7d660, 0xc0001b7500}, 0xc000182b00)
	D:/Go/src/net/http/server.go:2936 +0x683
net/http.(*conn).serve(0xc0001501b0, {0xd7db38, 0xc001ad8150})
	D:/Go/src/net/http/server.go:1995 +0xbd5
created by net/http.(*Server).Serve
	D:/Go/src/net/http/server.go:3089 +0x818

goroutine 1 [chan receive]:
main.main()
	F:/netsvr/cmd/netsvr.go:38 +0x93

goroutine 50 [chan receive]:
github.com/lesismal/nbio/taskpool.(*FixedNoOrderPool).taskLoop(0xc00009a000)
	F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:13 +0x6b
created by github.com/lesismal/nbio/taskpool.NewFixedNoOrderPool
	F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:40 +0x91

goroutine 7 [chan receive]:
github.com/lesismal/nbio/taskpool.(*FixedNoOrderPool).taskLoop(0xc00009a008)
	F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:13 +0x6b
created by github.com/lesismal/nbio/taskpool.NewFixedNoOrderPool
	F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:40 +0x91

goroutine 35 [chan receive]:
github.com/rcrowley/go-metrics.(*meterArbiter).tick(0x10ed6a0)
	F:/netsvr/vendor/github.com/rcrowley/go-metrics/meter.go:239 +0x5c
created by github.com/rcrowley/go-metrics.NewMeter
	F:/netsvr/vendor/github.com/rcrowley/go-metrics/meter.go:46 +0x171

goroutine 36 [chan receive]:
netsvr/internal/metrics.init.0.func2()
	F:/netsvr/internal/metrics/metrics.go:83 +0x10f
created by netsvr/internal/metrics.init.0
	F:/netsvr/internal/metrics/metrics.go:78 +0x726

goroutine 37 [syscall]:
os/signal.signal_recv()
	D:/Go/src/runtime/sigqueue.go:152 +0x2f
os/signal.loop()
	D:/Go/src/os/signal/signal_unix.go:23 +0x25
created by os/signal.Notify.func1.1
	D:/Go/src/os/signal/signal.go:151 +0x51

goroutine 38 [chan receive]:
netsvr/pkg/quit.init.1.func1()
	F:/netsvr/pkg/quit/quit.go:62 +0x45
created by netsvr/pkg/quit.init.1
	F:/netsvr/pkg/quit/quit.go:61 +0x118

goroutine 39 [select]:
github.com/antlabs/timer.(*timeWheel).Run(0xc0000c3300)
	F:/netsvr/vendor/github.com/antlabs/timer/time_wheel.go:293 +0x185
created by netsvr/internal/timer.init.0
	F:/netsvr/internal/timer/timer.go:30 +0x10a

goroutine 40 [chan receive]:
netsvr/internal/timer.init.0.func1()
	F:/netsvr/internal/timer/timer.go:36 +0x59
created by netsvr/internal/timer.init.0
	F:/netsvr/internal/timer/timer.go:32 +0x116

goroutine 41 [IO wait]:
internal/poll.runtime_pollWait(0x2a2a0e18, 0x72)
	D:/Go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc0001fe438, 0x10ddb80?, 0x0)
	D:/Go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.execIO(0xc0001fe298, 0xc0001f9960)
	D:/Go/src/internal/poll/fd_windows.go:175 +0x285
internal/poll.(*FD).acceptOne(0xc0001fe280, 0x204, {0xc000a202d0, 0x2, 0x2}, 0xc0001fe298)
	D:/Go/src/internal/poll/fd_windows.go:936 +0xda
internal/poll.(*FD).Accept(0xc0001fe280, 0xc0001f9bc8)
	D:/Go/src/internal/poll/fd_windows.go:970 +0x308
net.(*netFD).accept(0xc0001fe280)
	D:/Go/src/net/fd_windows.go:139 +0x75
net.(*TCPListener).accept(0xc000198078)
	D:/Go/src/net/tcpsock_posix.go:148 +0x45
net.(*TCPListener).Accept(0xc000198078)
	D:/Go/src/net/tcpsock.go:297 +0x68
net/http.(*Server).Serve(0xc0001b80f0, {0xd7d480, 0xc000198078})
	D:/Go/src/net/http/server.go:3059 +0x5a7
net/http.(*Server).ListenAndServe(0xc0001b80f0)
	D:/Go/src/net/http/server.go:2988 +0xc5
net/http.ListenAndServe(...)
	D:/Go/src/net/http/server.go:3242
main.pprof.func1()
	F:/netsvr/cmd/netsvr.go:67 +0x20c
created by main.pprof
	F:/netsvr/cmd/netsvr.go:62 +0x2a

goroutine 42 [IO wait]:
internal/poll.runtime_pollWait(0x2a2a0f08, 0x72)
	D:/Go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc000a1c1b8, 0xc0000679d8?, 0x0)
	D:/Go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.execIO(0xc000a1c018, 0xc000067a60)
	D:/Go/src/internal/poll/fd_windows.go:175 +0x285
internal/poll.(*FD).acceptOne(0xc000a1c000, 0x1b0, {0xc000a20000, 0x2, 0x2}, 0xc000a1c018)
	D:/Go/src/internal/poll/fd_windows.go:936 +0xda
internal/poll.(*FD).Accept(0xc000a1c000, 0xc000067cc8)
	D:/Go/src/internal/poll/fd_windows.go:970 +0x308
net.(*netFD).accept(0xc000a1c000)
	D:/Go/src/net/fd_windows.go:139 +0x75
net.(*TCPListener).accept(0xc000a0a030)
	D:/Go/src/net/tcpsock_posix.go:148 +0x45
net.(*TCPListener).Accept(0xc000a0a030)
	D:/Go/src/net/tcpsock.go:297 +0x68
netsvr/internal/worker.(*Server).Start(0xc000a16010)
	F:/netsvr/internal/worker/tcp.go:43 +0xa6
netsvr/internal/worker.Start()
	F:/netsvr/internal/worker/tcp.go:125 +0x13d
created by main.main
	F:/netsvr/cmd/netsvr.go:35 +0x68

goroutine 20 [chan receive]:
gopkg.in/natefinch/lumberjack%2ev2.(*Logger).millRun(0xc00005e240)
	F:/netsvr/vendor/gopkg.in/natefinch/lumberjack.v2/lumberjack.go:379 +0x66
created by gopkg.in/natefinch/lumberjack%2ev2.(*Logger).mill.func1
	F:/netsvr/vendor/gopkg.in/natefinch/lumberjack.v2/lumberjack.go:390 +0xef

goroutine 51 [chan receive]:
github.com/lesismal/nbio/taskpool.(*FixedNoOrderPool).taskLoop(0xc00009a018)
	F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:13 +0x6b
created by github.com/lesismal/nbio/taskpool.NewFixedNoOrderPool
	F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:40 +0x91

goroutine 52 [chan receive]:
github.com/lesismal/nbio/taskpool.(*FixedNoOrderPool).taskLoop(0xc00009a030)
	F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:13 +0x6b
created by github.com/lesismal/nbio/taskpool.NewFixedNoOrderPool
	F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:40 +0x91

goroutine 53 [chan receive]:
github.com/lesismal/nbio.(*poller).start(0xc000a06300)
	F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:122 +0xa25
created by github.com/lesismal/nbio.(*Engine).Start
	F:/netsvr/vendor/github.com/lesismal/nbio/engine_std.go:73 +0x2f2

goroutine 54 [chan receive]:
github.com/lesismal/nbio.(*poller).start(0xc000a06360)
	F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:122 +0xa25
created by github.com/lesismal/nbio.(*Engine).Start
	F:/netsvr/vendor/github.com/lesismal/nbio/engine_std.go:73 +0x2f2

goroutine 55 [chan receive]:
github.com/lesismal/nbio.(*poller).start(0xc000a063c0)
	F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:122 +0xa25
created by github.com/lesismal/nbio.(*Engine).Start
	F:/netsvr/vendor/github.com/lesismal/nbio/engine_std.go:73 +0x2f2

goroutine 56 [chan receive]:
github.com/lesismal/nbio.(*poller).start(0xc000a06420)
	F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:122 +0xa25
created by github.com/lesismal/nbio.(*Engine).Start
	F:/netsvr/vendor/github.com/lesismal/nbio/engine_std.go:73 +0x2f2

goroutine 57 [select]:
github.com/lesismal/nbio/timer.(*Timer).loop(0xc000090100)
	F:/netsvr/vendor/github.com/lesismal/nbio/timer/timer.go:154 +0x345
created by github.com/lesismal/nbio/timer.(*Timer).Start
	F:/netsvr/vendor/github.com/lesismal/nbio/timer/timer.go:57 +0xa7

goroutine 58 [IO wait]:
internal/poll.runtime_pollWait(0x2a2a0d28, 0x72)
	D:/Go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc000a1c438, 0x114c060?, 0x0)
	D:/Go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.execIO(0xc000a1c298, 0xc000a23bb0)
	D:/Go/src/internal/poll/fd_windows.go:175 +0x285
internal/poll.(*FD).acceptOne(0xc000a1c280, 0x1f0, {0xc000a200f0, 0x2, 0x2}, 0xc000a1c298)
	D:/Go/src/internal/poll/fd_windows.go:936 +0xda
internal/poll.(*FD).Accept(0xc000a1c280, 0xc000a23e18)
	D:/Go/src/internal/poll/fd_windows.go:970 +0x308
net.(*netFD).accept(0xc000a1c280)
	D:/Go/src/net/fd_windows.go:139 +0x75
net.(*TCPListener).accept(0xc000a04168)
	D:/Go/src/net/tcpsock_posix.go:148 +0x45
net.(*TCPListener).Accept(0xc000a04168)
	D:/Go/src/net/tcpsock.go:297 +0x68
github.com/lesismal/nbio/nbhttp.(*Engine).listen.func1()
	F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:271 +0x11d
created by github.com/lesismal/nbio/nbhttp.(*Engine).listen
	F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:265 +0x1cf

goroutine 60 [IO wait]:
internal/poll.runtime_pollWait(0x2a2a0b48, 0x72)
	D:/Go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc000a1c938, 0x0?, 0x0)
	D:/Go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.execIO(0xc000a1c798, 0xcbbb08)
	D:/Go/src/internal/poll/fd_windows.go:175 +0x285
internal/poll.(*FD).Read(0xc000a1c780, {0xc000a3c000, 0x1000, 0x1000})
	D:/Go/src/internal/poll/fd_windows.go:436 +0x445
net.(*netFD).Read(0xc000a1c780, {0xc000a3c000, 0x1000, 0x1000})
	D:/Go/src/net/fd_posix.go:55 +0x51
net.(*conn).Read(0xc00009a048, {0xc000a3c000, 0x1000, 0x1000})
	D:/Go/src/net/net.go:183 +0xb1
net/http.(*connReader).Read(0xc000031590, {0xc000a3c000, 0x1000, 0x1000})
	D:/Go/src/net/http/server.go:782 +0x23b
bufio.(*Reader).fill(0xc000a066c0)
	D:/Go/src/bufio/bufio.go:106 +0x2ab
bufio.(*Reader).ReadSlice(0xc000a066c0, 0xd0?)
	D:/Go/src/bufio/bufio.go:372 +0x89
bufio.(*Reader).ReadLine(0xc000a066c0)
	D:/Go/src/bufio/bufio.go:401 +0x36
net/textproto.(*Reader).readLineSlice(0xc0000315c0)
	D:/Go/src/net/textproto/reader.go:54 +0xe5
net/textproto.(*Reader).ReadLine(...)
	D:/Go/src/net/textproto/reader.go:37
net/http.readRequest(0xc00009a048?)
	D:/Go/src/net/http/request.go:1042 +0xd3
net/http.(*conn).readRequest(0xc000150240, {0xd7da90, 0xc000060320})
	D:/Go/src/net/http/server.go:990 +0x5aa
net/http.(*conn).serve(0xc000150240, {0xd7db38, 0xc001ad8150})
	D:/Go/src/net/http/server.go:1920 +0x5d4
created by net/http.(*Server).Serve
	D:/Go/src/net/http/server.go:3089 +0x818

goroutine 49 [IO wait]:
internal/poll.runtime_pollWait(0x2a2a0c38, 0x72)
	D:/Go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc000a1c6b8, 0x2?, 0x0)
	D:/Go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.execIO(0xc000a1c518, 0xcbbb08)
	D:/Go/src/internal/poll/fd_windows.go:175 +0x285
internal/poll.(*FD).Read(0xc000a1c500, {0xc0000314b1, 0x1, 0x1})
	D:/Go/src/internal/poll/fd_windows.go:436 +0x445
net.(*netFD).Read(0xc000a1c500, {0xc0000314b1, 0x1, 0x1})
	D:/Go/src/net/fd_posix.go:55 +0x51
net.(*conn).Read(0xc00009a040, {0xc0000314b1, 0x1, 0x1})
	D:/Go/src/net/net.go:183 +0xb1
net/http.(*connReader).backgroundRead(0xc0000314a0)
	D:/Go/src/net/http/server.go:674 +0x79
created by net/http.(*connReader).startBackgroundRead
	D:/Go/src/net/http/server.go:670 +0x1a7

大量连接并发进来后，再并发退出后的服务端协程堆栈

goroutine 59 [running]:
runtime/pprof.writeGoroutineStacks({0xd7a240, 0xc003da6700})
	D:/Go/src/runtime/pprof/pprof.go:703 +0x85
runtime/pprof.writeGoroutine({0xd7a240, 0xc003da6700}, 0x2)
	D:/Go/src/runtime/pprof/pprof.go:692 +0x4b
runtime/pprof.(*Profile).WriteTo(0x10e0f60, {0xd7a240, 0xc003da6700}, 0xc?)
	D:/Go/src/runtime/pprof/pprof.go:329 +0x1bb
net/http/pprof.handler.ServeHTTP({0xc0055c8521, 0x9}, {0xd7d660, 0xc003da6700}, 0x0?)
	D:/Go/src/net/http/pprof/pprof.go:259 +0x57f
net/http/pprof.Index({0xd7d660?, 0xc003da6700}, 0xc005891e00)
	D:/Go/src/net/http/pprof/pprof.go:376 +0x12c
net/http.HandlerFunc.ServeHTTP(0xcbbfb8, {0xd7d660, 0xc003da6700}, 0x0?)
	D:/Go/src/net/http/server.go:2122 +0x4e
net/http.(*ServeMux).ServeHTTP(0xc0055c852b?, {0xd7d660, 0xc003da6700}, 0xc005891e00)
	D:/Go/src/net/http/server.go:2500 +0xc6
net/http.serverHandler.ServeHTTP({0xc0000314a0?}, {0xd7d660, 0xc003da6700}, 0xc005891e00)
	D:/Go/src/net/http/server.go:2936 +0x683
net/http.(*conn).serve(0xc0001501b0, {0xd7db38, 0xc001ad8150})
	D:/Go/src/net/http/server.go:1995 +0xbd5
created by net/http.(*Server).Serve
	D:/Go/src/net/http/server.go:3089 +0x818

goroutine 1 [chan receive, 2 minutes]:
main.main()
	F:/netsvr/cmd/netsvr.go:38 +0x93

goroutine 50 [chan receive, 2 minutes]:
github.com/lesismal/nbio/taskpool.(*FixedNoOrderPool).taskLoop(0xc00009a000)
	F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:13 +0x6b
created by github.com/lesismal/nbio/taskpool.NewFixedNoOrderPool
	F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:40 +0x91

goroutine 7 [chan receive, 2 minutes]:
github.com/lesismal/nbio/taskpool.(*FixedNoOrderPool).taskLoop(0xc00009a008)
	F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:13 +0x6b
created by github.com/lesismal/nbio/taskpool.NewFixedNoOrderPool
	F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:40 +0x91

goroutine 35 [chan receive]:
github.com/rcrowley/go-metrics.(*meterArbiter).tick(0x10ed6a0)
	F:/netsvr/vendor/github.com/rcrowley/go-metrics/meter.go:239 +0x5c
created by github.com/rcrowley/go-metrics.NewMeter
	F:/netsvr/vendor/github.com/rcrowley/go-metrics/meter.go:46 +0x171

goroutine 36 [chan receive]:
netsvr/internal/metrics.init.0.func2()
	F:/netsvr/internal/metrics/metrics.go:83 +0x10f
created by netsvr/internal/metrics.init.0
	F:/netsvr/internal/metrics/metrics.go:78 +0x726

goroutine 37 [syscall, 2 minutes]:
os/signal.signal_recv()
	D:/Go/src/runtime/sigqueue.go:152 +0x2f
os/signal.loop()
	D:/Go/src/os/signal/signal_unix.go:23 +0x25
created by os/signal.Notify.func1.1
	D:/Go/src/os/signal/signal.go:151 +0x51

goroutine 38 [chan receive, 2 minutes]:
netsvr/pkg/quit.init.1.func1()
	F:/netsvr/pkg/quit/quit.go:62 +0x45
created by netsvr/pkg/quit.init.1
	F:/netsvr/pkg/quit/quit.go:61 +0x118

goroutine 39 [select]:
github.com/antlabs/timer.(*timeWheel).Run(0xc0000c3300)
	F:/netsvr/vendor/github.com/antlabs/timer/time_wheel.go:293 +0x185
created by netsvr/internal/timer.init.0
	F:/netsvr/internal/timer/timer.go:30 +0x10a

goroutine 40 [chan receive, 2 minutes]:
netsvr/internal/timer.init.0.func1()
	F:/netsvr/internal/timer/timer.go:36 +0x59
created by netsvr/internal/timer.init.0
	F:/netsvr/internal/timer/timer.go:32 +0x116

goroutine 41 [IO wait]:
internal/poll.runtime_pollWait(0x2a2a0e18, 0x72)
	D:/Go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc0001fe438, 0x10ddb80?, 0x0)
	D:/Go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.execIO(0xc0001fe298, 0xc002983960)
	D:/Go/src/internal/poll/fd_windows.go:175 +0x285
internal/poll.(*FD).acceptOne(0xc0001fe280, 0x984, {0xc004af0690, 0x2, 0x2}, 0xc0001fe298)
	D:/Go/src/internal/poll/fd_windows.go:936 +0xda
internal/poll.(*FD).Accept(0xc0001fe280, 0xc003b95bc8)
	D:/Go/src/internal/poll/fd_windows.go:970 +0x308
net.(*netFD).accept(0xc0001fe280)
	D:/Go/src/net/fd_windows.go:139 +0x75
net.(*TCPListener).accept(0xc000198078)
	D:/Go/src/net/tcpsock_posix.go:148 +0x45
net.(*TCPListener).Accept(0xc000198078)
	D:/Go/src/net/tcpsock.go:297 +0x68
net/http.(*Server).Serve(0xc0001b80f0, {0xd7d480, 0xc000198078})
	D:/Go/src/net/http/server.go:3059 +0x5a7
net/http.(*Server).ListenAndServe(0xc0001b80f0)
	D:/Go/src/net/http/server.go:2988 +0xc5
net/http.ListenAndServe(...)
	D:/Go/src/net/http/server.go:3242
main.pprof.func1()
	F:/netsvr/cmd/netsvr.go:67 +0x20c
created by main.pprof
	F:/netsvr/cmd/netsvr.go:62 +0x2a

goroutine 42 [IO wait, 2 minutes]:
internal/poll.runtime_pollWait(0x2a2a0f08, 0x72)
	D:/Go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc000a1c1b8, 0x114c060?, 0x0)
	D:/Go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.execIO(0xc000a1c018, 0xc000067a60)
	D:/Go/src/internal/poll/fd_windows.go:175 +0x285
internal/poll.(*FD).acceptOne(0xc000a1c000, 0x21c, {0xc001ae00f0, 0x2, 0x2}, 0xc000a1c018)
	D:/Go/src/internal/poll/fd_windows.go:936 +0xda
internal/poll.(*FD).Accept(0xc000a1c000, 0xc002985cc8)
	D:/Go/src/internal/poll/fd_windows.go:970 +0x308
net.(*netFD).accept(0xc000a1c000)
	D:/Go/src/net/fd_windows.go:139 +0x75
net.(*TCPListener).accept(0xc000a0a030)
	D:/Go/src/net/tcpsock_posix.go:148 +0x45
net.(*TCPListener).Accept(0xc000a0a030)
	D:/Go/src/net/tcpsock.go:297 +0x68
netsvr/internal/worker.(*Server).Start(0xc000a16010)
	F:/netsvr/internal/worker/tcp.go:43 +0xa6
netsvr/internal/worker.Start()
	F:/netsvr/internal/worker/tcp.go:125 +0x13d
created by main.main
	F:/netsvr/cmd/netsvr.go:35 +0x68

goroutine 20 [chan receive, 2 minutes]:
gopkg.in/natefinch/lumberjack%2ev2.(*Logger).millRun(0xc00005e240)
	F:/netsvr/vendor/gopkg.in/natefinch/lumberjack.v2/lumberjack.go:379 +0x66
created by gopkg.in/natefinch/lumberjack%2ev2.(*Logger).mill.func1
	F:/netsvr/vendor/gopkg.in/natefinch/lumberjack.v2/lumberjack.go:390 +0xef

goroutine 51 [chan receive, 2 minutes]:
github.com/lesismal/nbio/taskpool.(*FixedNoOrderPool).taskLoop(0xc00009a018)
	F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:13 +0x6b
created by github.com/lesismal/nbio/taskpool.NewFixedNoOrderPool
	F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:40 +0x91

goroutine 52 [chan receive, 2 minutes]:
github.com/lesismal/nbio/taskpool.(*FixedNoOrderPool).taskLoop(0xc00009a030)
	F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:13 +0x6b
created by github.com/lesismal/nbio/taskpool.NewFixedNoOrderPool
	F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:40 +0x91

goroutine 53 [chan receive, 2 minutes]:
github.com/lesismal/nbio.(*poller).start(0xc000a06300)
	F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:122 +0xa25
created by github.com/lesismal/nbio.(*Engine).Start
	F:/netsvr/vendor/github.com/lesismal/nbio/engine_std.go:73 +0x2f2

goroutine 54 [chan receive, 2 minutes]:
github.com/lesismal/nbio.(*poller).start(0xc000a06360)
	F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:122 +0xa25
created by github.com/lesismal/nbio.(*Engine).Start
	F:/netsvr/vendor/github.com/lesismal/nbio/engine_std.go:73 +0x2f2

goroutine 55 [chan receive, 2 minutes]:
github.com/lesismal/nbio.(*poller).start(0xc000a063c0)
	F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:122 +0xa25
created by github.com/lesismal/nbio.(*Engine).Start
	F:/netsvr/vendor/github.com/lesismal/nbio/engine_std.go:73 +0x2f2

goroutine 56 [chan receive, 2 minutes]:
github.com/lesismal/nbio.(*poller).start(0xc000a06420)
	F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:122 +0xa25
created by github.com/lesismal/nbio.(*Engine).Start
	F:/netsvr/vendor/github.com/lesismal/nbio/engine_std.go:73 +0x2f2

goroutine 57 [select, 2 minutes]:
github.com/lesismal/nbio/timer.(*Timer).loop(0xc000090100)
	F:/netsvr/vendor/github.com/lesismal/nbio/timer/timer.go:154 +0x345
created by github.com/lesismal/nbio/timer.(*Timer).Start
	F:/netsvr/vendor/github.com/lesismal/nbio/timer/timer.go:57 +0xa7

goroutine 58 [IO wait]:
internal/poll.runtime_pollWait(0x2a2a0d28, 0x72)
	D:/Go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc000a1c438, 0xc000a23b20?, 0x0)
	D:/Go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.execIO(0xc000a1c298, 0xc000a23bb0)
	D:/Go/src/internal/poll/fd_windows.go:175 +0x285
internal/poll.(*FD).acceptOne(0xc000a1c280, 0xa08, {0xc003f6e3c0, 0x2, 0x2}, 0xc000a1c298)
	D:/Go/src/internal/poll/fd_windows.go:936 +0xda
internal/poll.(*FD).Accept(0xc000a1c280, 0xc000a23e18)
	D:/Go/src/internal/poll/fd_windows.go:970 +0x308
net.(*netFD).accept(0xc000a1c280)
	D:/Go/src/net/fd_windows.go:139 +0x75
net.(*TCPListener).accept(0xc000a04168)
	D:/Go/src/net/tcpsock_posix.go:148 +0x45
net.(*TCPListener).Accept(0xc000a04168)
	D:/Go/src/net/tcpsock.go:297 +0x68
github.com/lesismal/nbio/nbhttp.(*Engine).listen.func1()
	F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:271 +0x11d
created by github.com/lesismal/nbio/nbhttp.(*Engine).listen
	F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:265 +0x1cf

goroutine 7665 [IO wait]:
internal/poll.runtime_pollWait(0x2a2a0a58, 0x72)
	D:/Go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc00417ce38, 0xc000092ea0?, 0x0)
	D:/Go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.execIO(0xc00417cc98, 0xcbbb08)
	D:/Go/src/internal/poll/fd_windows.go:175 +0x285
internal/poll.(*FD).Read(0xc00417cc80, {0xc0023f0000, 0x1000, 0x1000})
	D:/Go/src/internal/poll/fd_windows.go:436 +0x445
net.(*netFD).Read(0xc00417cc80, {0xc0023f0000, 0x1000, 0x1000})
	D:/Go/src/net/fd_posix.go:55 +0x51
net.(*conn).Read(0xc000a6c398, {0xc0023f0000, 0x1000, 0x1000})
	D:/Go/src/net/net.go:183 +0xb1
net/http.(*connReader).Read(0xc005a4b230, {0xc0023f0000, 0x1000, 0x1000})
	D:/Go/src/net/http/server.go:782 +0x23b
bufio.(*Reader).fill(0xc0058ce960)
	D:/Go/src/bufio/bufio.go:106 +0x2ab
bufio.(*Reader).ReadSlice(0xc0058ce960, 0xd0?)
	D:/Go/src/bufio/bufio.go:372 +0x89
bufio.(*Reader).ReadLine(0xc0058ce960)
	D:/Go/src/bufio/bufio.go:401 +0x36
net/textproto.(*Reader).readLineSlice(0xc005a4b260)
	D:/Go/src/net/textproto/reader.go:54 +0xe5
net/textproto.(*Reader).ReadLine(...)
	D:/Go/src/net/textproto/reader.go:37
net/http.readRequest(0xc000a6c398?)
	D:/Go/src/net/http/request.go:1042 +0xd3
net/http.(*conn).readRequest(0xc0056f3b90, {0xd7da90, 0xc000a70b40})
	D:/Go/src/net/http/server.go:990 +0x5aa
net/http.(*conn).serve(0xc0056f3b90, {0xd7db38, 0xc001ad8150})
	D:/Go/src/net/http/server.go:1920 +0x5d4
created by net/http.(*Server).Serve
	D:/Go/src/net/http/server.go:3089 +0x818

goroutine 7738 [IO wait]:
internal/poll.runtime_pollWait(0x2a2a0c38, 0x72)
	D:/Go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc000a1c6b8, 0x2?, 0x0)
	D:/Go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.execIO(0xc000a1c518, 0xcbbb08)
	D:/Go/src/internal/poll/fd_windows.go:175 +0x285
internal/poll.(*FD).Read(0xc000a1c500, {0xc0000314b1, 0x1, 0x1})
	D:/Go/src/internal/poll/fd_windows.go:436 +0x445
net.(*netFD).Read(0xc000a1c500, {0xc0000314b1, 0x1, 0x1})
	D:/Go/src/net/fd_posix.go:55 +0x51
net.(*conn).Read(0xc00009a040, {0xc0000314b1, 0x1, 0x1})
	D:/Go/src/net/net.go:183 +0xb1
net/http.(*connReader).backgroundRead(0xc0000314a0)
	D:/Go/src/net/http/server.go:674 +0x79
created by net/http.(*connReader).startBackgroundRead
	D:/Go/src/net/http/server.go:670 +0x1a7

//就是这条协程，我感觉它是泄漏的了。
goroutine 6173 [IO wait]:
internal/poll.runtime_pollWait(0x2a2d21f8, 0x72)
	D:/Go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc003fbebb8, 0xa34ab3?, 0x0)
	D:/Go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.execIO(0xc003fbea18, 0xcbbb08)
	D:/Go/src/internal/poll/fd_windows.go:175 +0x285
internal/poll.(*FD).Read(0xc003fbea00, {0xc00526a000, 0x8000, 0x8000})
	D:/Go/src/internal/poll/fd_windows.go:436 +0x445
net.(*netFD).Read(0xc003fbea00, {0xc00526a000, 0x8000, 0x8000})
	D:/Go/src/net/fd_posix.go:55 +0x51
net.(*conn).Read(0xc0051f8178, {0xc00526a000, 0x8000, 0x8000})
	D:/Go/src/net/net.go:183 +0xb1
github.com/lesismal/nbio.(*Conn).readTCP(0xc0051e2460, {0xc00526a000, 0x8000, 0x8000})
	F:/netsvr/vendor/github.com/lesismal/nbio/conn_std.go:108 +0xd0
github.com/lesismal/nbio.(*Conn).read(0xc0051e2460, {0xc00526a000, 0x8000, 0x8000})
	F:/netsvr/vendor/github.com/lesismal/nbio/conn_std.go:95 +0x7b
github.com/lesismal/nbio.(*poller).readConn(0xc000a06360, 0xc0050afe78?)
	F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:57 +0x8e
created by github.com/lesismal/nbio.(*poller).addConn
	F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:77 +0x205

竞争警告

==================
WARNING: DATA RACE                                                         
Read at 0x00c0051e24a0 by goroutine 328:                                   
  github.com/lesismal/nbio.(*poller).deleteConn()                          
      F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:89 +0xdb     
  github.com/lesismal/nbio.(*Conn).Close()                                 
      F:/netsvr/vendor/github.com/lesismal/nbio/conn_std.go:275 +0x1c7     
  github.com/lesismal/nbio/nbhttp.(*Engine).Shutdown.func2()               
      F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:441 +0x44 
                                                                           
Previous write at 0x00c0051e24a0 by goroutine 763:                         
  github.com/lesismal/nbio.(*Conn).readTCP()                               
      F:/netsvr/vendor/github.com/lesismal/nbio/conn_std.go:110 +0x10b     
  github.com/lesismal/nbio.(*Conn).read()                                  
      F:/netsvr/vendor/github.com/lesismal/nbio/conn_std.go:95 +0x7a       
  github.com/lesismal/nbio.(*poller).readConn()                            
      F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:57 +0x8d     
  github.com/lesismal/nbio.(*poller).addConn.func1()                       
      F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:77 +0x47     
                                                                           
Goroutine 328 (running) created at:                                        
  github.com/lesismal/nbio/nbhttp.(*Engine).Shutdown()                     
      F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:439 +0x251
  netsvr/internal/customer.Shutdown()                                      
      F:/netsvr/internal/customer/websocket.go:72 +0x74
  main.main()
      F:/netsvr/cmd/netsvr.go:55 +0x17a

Goroutine 763 (running) created at:
  github.com/lesismal/nbio.(*poller).addConn()
      F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:77 +0x204
  github.com/lesismal/nbio.(*Engine).AddConn()
      F:/netsvr/vendor/github.com/lesismal/nbio/engine.go:211 +0x10d
  github.com/lesismal/nbio/nbhttp.(*Engine).AddConnNonTLSNonBlocking()
      F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:587 +0x5ad
  github.com/lesismal/nbio/nbhttp.(*Engine).AddConnNonTLSNonBlocking-fm()
      <autogenerated>:1 +0x6d
  github.com/lesismal/nbio/nbhttp.(*Engine).listen.func1()
      F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:273 +0x250
==================
==================
WARNING: DATA RACE
Read at 0x00c000031290 by main goroutine:
  github.com/lesismal/nbio/nbhttp.(*Engine).Shutdown()
      F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:453 +0x45d
  netsvr/internal/customer.Shutdown()
      F:/netsvr/internal/customer/websocket.go:72 +0x74
  main.main()
      F:/netsvr/cmd/netsvr.go:55 +0x17a

Previous write at 0x00c000031290 by goroutine 600:
  runtime.mapdelete_fast64()
      D:/Go/src/runtime/map_fast64.go:273 +0x0
  github.com/lesismal/nbio/nbhttp.NewEngine.func5.1()
      F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:936 +0x10f
  github.com/lesismal/nbio.(*Conn).MustExecute.func1()
      F:/netsvr/vendor/github.com/lesismal/nbio/conn.go:157 +0x65
  github.com/lesismal/nbio/taskpool.(*MixedPool).callWitoutRecover()
      F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/mixedpool.go:38 +0x72
  github.com/lesismal/nbio/taskpool.(*MixedPool).callWitoutRecover-fm()
      <autogenerated>:1 +0x44
  github.com/lesismal/nbio/taskpool.(*MixedPool).Go.func1()
      F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/mixedpool.go:45 +0x77

Goroutine 600 (finished) created at:
  github.com/lesismal/nbio/taskpool.(*MixedPool).Go()
      F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/mixedpool.go:44 +0x15e
  github.com/lesismal/nbio/taskpool.(*MixedPool).Go-fm()
      <autogenerated>:1 +0x44
  github.com/lesismal/nbio.(*Conn).MustExecute()
      F:/netsvr/vendor/github.com/lesismal/nbio/conn.go:154 +0x321
  github.com/lesismal/nbio/nbhttp.NewEngine.func5()
      F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:929 +0x125
  github.com/lesismal/nbio.(*Engine).OnClose.func1()
      F:/netsvr/vendor/github.com/lesismal/nbio/engine.go:234 +0xb5
  github.com/lesismal/nbio.(*poller).deleteConn()
      F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:89 +0x13d
  github.com/lesismal/nbio.(*Conn).Close()
      F:/netsvr/vendor/github.com/lesismal/nbio/conn_std.go:275 +0x1c7
  github.com/lesismal/nbio/nbhttp.(*Engine).Shutdown.func2()
      F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:441 +0x44
==================
Found 2 data race(s)

Process finished with the exit code 66

lesismal · 2023-03-15T09:54:23Z

//就是这条协程，我感觉它是泄漏的了。
goroutine 6173 [IO wait]:
internal/poll.runtime_pollWait(0x2a2d21f8, 0x72)
D:/Go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc003fbebb8, 0xa34ab3?, 0x0)
D:/Go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.execIO(0xc003fbea18, 0xcbbb08)
D:/Go/src/internal/poll/fd_windows.go:175 +0x285
internal/poll.(*FD).Read(0xc003fbea00, {0xc00526a000, 0x8000, 0x8000})
D:/Go/src/internal/poll/fd_windows.go:436 +0x445
net.(*netFD).Read(0xc003fbea00, {0xc00526a000, 0x8000, 0x8000})
D:/Go/src/net/fd_posix.go:55 +0x51
net.(*conn).Read(0xc0051f8178, {0xc00526a000, 0x8000, 0x8000})
D:/Go/src/net/net.go:183 +0xb1
github.com/lesismal/nbio.(*Conn).readTCP(0xc0051e2460, {0xc00526a000, 0x8000, 0x8000})
F:/netsvr/vendor/github.com/lesismal/nbio/conn_std.go:108 +0xd0
github.com/lesismal/nbio.(*Conn).read(0xc0051e2460, {0xc00526a000, 0x8000, 0x8000})
F:/netsvr/vendor/github.com/lesismal/nbio/conn_std.go:95 +0x7b
github.com/lesismal/nbio.(*poller).readConn(0xc000a06360, 0xc0050afe78?)
F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:57 +0x8e
created by github.com/lesismal/nbio.(*poller).addConn
F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:77 +0x205

看样子像是windows的读阻塞了，如果是client side 退出通常会 TCP FIN，这里读应该就会返回。如果是server主动close，我以前好像也遇到过标准库net.TCPConn.Close并没有触发读err的问题，如果是server主动close的情况导致，以前项目里是Close时也调用了CloseRead/CloseWrite应该就可以，所以我考虑在这里加上CloseRead/CloseWrite来主动关闭了（但是这很不合理，因为是标准库的net.TCPConn，）：
https://github.com/lesismal/nbio/blob/master/conn_std.go#L266
修改：

case ConnTypeTCP:
	err = c.conn.(*net.TCPConn).Close()
	if err == nil {
		if err1 := c.conn.(*net.TCPConn).CloseRead(); err1 != nil {
			err = err1
		}
		if err2 := c.conn.(*net.TCPConn).CloseWrite(); err2 != nil {
			err = err2
		}
	}

有简单的完整代码可以复现这种泄露吗？我试试看什么问题。

uni*应该是没这个问题的，windows主要作为调试使用，应该不会影响linux下的server。

另外，server应该在连接建立后以及收到新的message后进行SetReadDeadline，则这个读也不会一直阻塞、超时后就会退出了。

buexplain · 2023-03-16T07:22:08Z

是客户端主动关闭的连接。
这个问题不是必现的，我尝试了写一个demo去复现它，结果不成功；我只能在我那个项目里面复现它。
Linux下协程数量稳定，没这个问题。

lesismal · 2023-03-16T08:51:46Z

我也写demo windows也尝试了下，遇到过一次一个连接Read阻塞，但是过了一会又返回err了，不确定是不是刚好赶巧了在多轮主动断开、重连的临界时间导致的，修改了一轮重连时间间隔长点之后避免这种临届时间后就没再遇到。

即使是Linux，也可能遇到client设备掉电，比如强行关机，或者手机进电梯地铁、切换信号塔之类的导致原来的连接断开但是没有收到TCP FIN、所以也会一直阻塞可能导致泄露。这些场景下，都是需要应用层通过ReadDeadline或者其他心跳检测的方式来实现server主动断开、这样才能确保不泄露。有了应用层保障，也就不怕这种偶发的问题。

而且从日志上看确实是runtime阻塞了，所以nbio本身暂时不修改这块代码了，后面如果能有办法稳定复现咱再来定位分析搞一波。

buexplain closed this as completed Mar 16, 2023

lesismal mentioned this issue Sep 9, 2023

WARNING: DATA RACE #346

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

数据并发读写冲突 #277

数据并发读写冲突 #277

buexplain commented Mar 15, 2023

lesismal commented Mar 15, 2023

buexplain commented Mar 15, 2023

lesismal commented Mar 15, 2023

buexplain commented Mar 16, 2023

lesismal commented Mar 16, 2023

数据并发读写冲突 #277

数据并发读写冲突 #277

Comments

buexplain commented Mar 15, 2023

lesismal commented Mar 15, 2023

buexplain commented Mar 15, 2023

lesismal commented Mar 15, 2023

buexplain commented Mar 16, 2023

lesismal commented Mar 16, 2023