CONNECT connections stuck waiting for EOF

[jameshartig]

I have thousands of stuck goroutines with this stack

#0  runtime.gopark (unlockf={void (struct runtime.g *, void *, bool *)} 0xc422033818, lock=0x7f34dd68cbb0, reason="IO wait", traceEv=27 '\033', traceskip=5) at /usr/local/go/src/runtime/proc.go:272
#1  0x0000000000429097 in runtime.netpollblock (pd=0x7f34dd68cb88, mode=114, waitio=false, ~r3=11) at /usr/local/go/src/runtime/netpoll.go:345
#2  0x00000000004286d9 in net.runtime_pollWait (pd=0x7f34dd68cb88, mode=114, ~r2=644) at /usr/local/go/src/runtime/netpoll.go:164
#3  0x0000000000559a08 in net.(*pollDesc).wait (pd=0xc421304618, mode=114, ~r1=...) at /usr/local/go/src/net/fd_poll_runtime.go:75
#4  0x0000000000559a74 in net.(*pollDesc).waitRead (pd=0xc421304618, ~r0=...) at /usr/local/go/src/net/fd_poll_runtime.go:80
#5  0x000000000055b1c7 in net.(*netFD).Read (fd=0xc4213045b0, p= []uint8 = {...}, n=0, err=...) at /usr/local/go/src/net/fd_unix.go:250
#6  0x000000000056e330 in net.(*conn).Read (c=0xc420162d28, b= []uint8 = {...}, ~r1=0, ~r2=...) at /usr/local/go/src/net/net.go:181
#7  0x00000000005b9c48 in crypto/tls.(*block).readFromUntil (b=0xc4220786c0, r=..., n=5, ~r2=...) at /usr/local/go/src/crypto/tls/conn.go:488
#8  0x00000000005ba1b4 in crypto/tls.(*Conn).readRecord (c=0xc421fa3c00, want=23 '\027', ~r1=...) at /usr/local/go/src/crypto/tls/conn.go:590
#9  0x00000000005bdebd in crypto/tls.(*Conn).Read (c=0xc421fa3c00, b= []uint8 = {...}, n#4=0, err=...) at /usr/local/go/src/crypto/tls/conn.go:1134
#10 0x00000000004caa97 in bufio.(*Reader).fill (b=0xc421a71d40) at /usr/local/go/src/bufio/bufio.go:97
#11 0x00000000004cac57 in bufio.(*Reader).Peek (b=0xc421a71d40, n=1, ~r1= []uint8, ~r2=...) at /usr/local/go/src/bufio/bufio.go:129
#12 0x00000000007e5278 in github.com/elazarl/goproxy.isEof (r=0xc421a71d40, ~r1=false) at /opt/TeamCityBuild/work/rachel-1/74d42b7cac97ce99/.deps/src/github.com/elazarl/goproxy/proxy.go:46
#13 0x00000000007e7c69 in github.com/elazarl/goproxy.(*ProxyHttpServer).handleHttps.func2 (proxyClient=..., tlsConfig=0xc42013a000, ctx#14=0xc42140ac30, r=0xc420bc7200, proxy=0xc420070e80)
    at /opt/TeamCityBuild/work/rachel-1/74d42b7cac97ce99/.deps/src/github.com/elazarl/goproxy/https.go:189
#14 0x000000000045a651 in runtime.goexit () at /usr/local/go/src/runtime/asm_amd64.s:2197
#15 0x0000000000ac9e60 in IPConn,net.Conn ()
#16 0x000000c420162d28 in ?? ()
#17 0x000000c42013a000 in ?? ()
#18 0x000000c42140ac30 in ?? ()
#19 0x000000c420bc7200 in ?? ()
#20 0x000000c420070e80 in ?? ()
#21 0x0000000000000000 in ?? ()

Which seems like they're stuck in the loop in https.go:

for !isEof(clientTlsReader) {

My code is pretty simple:

proxy.OnRequest().HandleConnectFunc(func(host string, _ *goproxy.ProxyCtx) (*goproxy.ConnectAction, string) {
    return &goproxy.ConnectAction{
        Action: goproxy.ConnectMitm,
        TLSConfig: func(_ string, _ *goproxy.ProxyCtx) (*tls.Config, error) {
            return tlsc, nil
        },
    }, host
})

The connections (proxy is port 7899) look like:

TCP 127.0.0.1:7899->127.0.0.1:39125 (ESTABLISHED)

I'm using the proxy with Chrome. Is this a Chrome bug that it's not closing the connection?

[elazarl]

Hi, It's certainly possible that Chrome is caching a few connections, and not closing them for future use. It does look suspicious that so much threads are parked. Unfortunately I do not have time to look deeply into this issue now, can you please try within a month or so? Thanks,

…

On Thu, Mar 16, 2017 at 2:41 AM, James Hartig ***@***.***> wrote: I have thousands of stuck goroutines with this stack #0 runtime.gopark (unlockf={void (struct runtime.g *, void *, bool *)} 0xc422033818, lock=0x7f34dd68cbb0, reason="IO wait", traceEv=27 '\033', traceskip=5) at /usr/local/go/src/runtime/proc.go:272 #1 0x0000000000429097 in runtime.netpollblock (pd=0x7f34dd68cb88, mode=114, waitio=false, ~r3=11) at /usr/local/go/src/runtime/netpoll.go:345 #2 0x00000000004286d9 in net.runtime_pollWait (pd=0x7f34dd68cb88, mode=114, ~r2=644) at /usr/local/go/src/runtime/netpoll.go:164 #3 0x0000000000559a08 in net.(*pollDesc).wait (pd=0xc421304618, mode=114, ~r1=...) at /usr/local/go/src/net/fd_poll_runtime.go:75 #4 0x0000000000559a74 in net.(*pollDesc).waitRead (pd=0xc421304618, ~r0=...) at /usr/local/go/src/net/fd_poll_runtime.go:80 #5 0x000000000055b1c7 in net.(*netFD).Read (fd=0xc4213045b0, p= []uint8 = {...}, n=0, err=...) at /usr/local/go/src/net/fd_unix.go:250 #6 0x000000000056e330 in net.(*conn).Read (c=0xc420162d28, b= []uint8 = {...}, ~r1=0, ~r2=...) at /usr/local/go/src/net/net.go:181 #7 0x00000000005b9c48 in crypto/tls.(*block).readFromUntil (b=0xc4220786c0, r=..., n=5, ~r2=...) at /usr/local/go/src/crypto/tls/conn.go:488 #8 0x00000000005ba1b4 in crypto/tls.(*Conn).readRecord (c=0xc421fa3c00, want=23 '\027', ~r1=...) at /usr/local/go/src/crypto/tls/conn.go:590 #9 0x00000000005bdebd in crypto/tls.(*Conn).Read (c=0xc421fa3c00, b= []uint8 = {...}, n#4=0, err=...) at /usr/local/go/src/crypto/tls/conn.go:1134 #10 0x00000000004caa97 in bufio.(*Reader).fill (b=0xc421a71d40) at /usr/local/go/src/bufio/bufio.go:97 #11 0x00000000004cac57 in bufio.(*Reader).Peek (b=0xc421a71d40, n=1, ~r1= []uint8, ~r2=...) at /usr/local/go/src/bufio/bufio.go:129 #12 0x00000000007e5278 in github.com/elazarl/goproxy.isEof (r=0xc421a71d40, ~r1=false) at /opt/TeamCityBuild/work/rachel-1/74d42b7cac97ce99/.deps/src/github.com/elazarl/goproxy/proxy.go:46 #13 <http://github.com/elazarl/goproxy/proxy.go:46#13> 0x00000000007e7c69 in github.com/elazarl/goproxy.(*ProxyHttpServer).handleHttps.func2 (proxyClient=..., tlsConfig=0xc42013a000, ctx#14=0xc42140ac30, r=0xc420bc7200, proxy=0xc420070e80) at /opt/TeamCityBuild/work/rachel-1/74d42b7cac97ce99/.deps/src/github.com/elazarl/goproxy/https.go:189 #14 <http://github.com/elazarl/goproxy/https.go:189#14> 0x000000000045a651 in runtime.goexit () at /usr/local/go/src/runtime/asm_amd64.s:2197 #15 0x0000000000ac9e60 in IPConn,net.Conn () #16 0x000000c420162d28 in ?? () #17 0x000000c42013a000 in ?? () #18 0x000000c42140ac30 in ?? () #19 0x000000c420bc7200 in ?? () #20 0x000000c420070e80 in ?? () #21 0x0000000000000000 in ?? () Which seems like they're stuck in the loop in https.go: for !isEof(clientTlsReader) { My code is pretty simple: proxy.OnRequest().HandleConnectFunc(func(host string, _ *goproxy.ProxyCtx) (*goproxy.ConnectAction, string) { return &goproxy.ConnectAction{ Action: goproxy.ConnectMitm, TLSConfig: func(_ string, _ *goproxy.ProxyCtx) (*tls.Config, error) { return tlsc, nil }, }, host }) The connections (proxy is port 7899) look like: TCP 127.0.0.1:7899->127.0.0.1:39125 (ESTABLISHED) I'm using the proxy with Chrome. Is this a Chrome bug that it's not closing the connection? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#209>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAP4ojUelUxyU07ZGpHQXi6NUhohmC-Sks5rmQPHgaJpZM4MfDCz> .