Skip to content

net/http: clients do not use full quota of MaxIdleConnsPerHost when called from multiple goroutines #24822

Closed
@ikenchina

Description

@ikenchina

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go 1.9.2

Does this issue reproduce with the latest release?

can't reproduce

What operating system and processor architecture are you using (go env)?

centos

scenario

create a new http client and default transport (client's timeout is 5s) in one goroutine to access http2 server every time.
but i have limit the amount of goroutine ( maximum is 100 goroutine ).
sometimes service didn't work, i found one goroutine was blocking others gorutines(99 goroutines).

blocked goroutines

in total, 98 goroutine have been blocked.

goroutine 220791540 [semacquire]: sync.runtime_SemacquireMutex(0xc429e00a14, 0xc427945500) /usr/local/go/src/runtime/sema.go:71 +0x3d sync.(*Mutex).Lock(0xc429e00a10) /usr/local/go/src/sync/mutex.go:134 +0xee net/http.(*http2ClientConn).streamByID(0xc429e009c0, 0x100000889, 0x0) /usr/local/go/src/net/http/h2_bundle.go:7758 +0x43 net/http.(*http2ClientConn).forgetStreamID(0xc429e009c0, 0x889) /usr/local/go/src/net/http/h2_bundle.go:7754 +0x38 net/http.(*http2ClientConn).RoundTrip(0xc429e009c0, 0xc425179200, 0x0, 0x0, 0x0) /usr/local/go/src/net/http/h2_bundle.go:7376 +0xb59 net/http.(*http2Transport).RoundTripOpt(0xc420837220, 0xc425179200, 0xc422d30600, 0xc425f546c0, 0x40000000040b130, 0xffffffffffffffff) /usr/local/go/src/net/http/h2_bundle.go:6858 +0x145 net/http.(*http2Transport).RoundTrip(0xc420837220, 0xc425179200, 0x654201, 0xc422d306f8, 0xc425f548f0) /usr/local/go/src/net/http/h2_bundle.go:6820 +0x3a net/http.http2noDialH2RoundTripper.RoundTrip(0xc420837220, 0xc425179200, 0xc426359f80, 0x5, 0xc420813048) /usr/local/go/src/net/http/h2_bundle.go:990 +0x39 net/http.(*Transport).RoundTrip(0x137f9a0, 0xc425179200, 0x137f9a0, 0xbea9f7ed1c00e6c5, 0x9a938b916ea9) /usr/local/go/src/net/http/transport.go:371 +0xd47 net/http.send(0xc425179100, 0x12f72e0, 0x137f9a0, 0xbea9f7ed1c00e6c5, 0x9a938b916ea9, 0x138d2a0, 0xc4201430d0, 0xbea9f7ed1c00e6c5, 0xc425f54b08, 0x1) /usr/local/go/src/net/http/client.go:249 +0x1a9 net/http.(*Client).send(0xc4208f15c0, 0xc425179100, 0xbea9f7ed1c00e6c5, 0x9a938b916ea9, 0x138d2a0, 0xc4201430d0, 0x0, 0x1, 0xbfb900) /usr/local/go/src/net/http/client.go:173 +0xfd net/http.(*Client).Do(0xc4208f15c0, 0xc425179100, 0x18, 0xc4200166c0, 0x18) /usr/local/go/src/net/http/client.go:602 +0x28d ......

blocking other goroutines

seems that this goroutine blocking other goroutines

goroutine 220791555 [IO wait]: internal/poll.runtime_pollWait(0x7f51f01de058, 0x77, 0x0) /usr/local/go/src/runtime/netpoll.go:173 +0x57 internal/poll.(*pollDesc).wait(0xc42545c098, 0x77, 0xffffffffffffff00, 0x12fa5e0, 0x12f46f0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0xae internal/poll.(*pollDesc).waitWrite(0xc42545c098, 0xc42264b800, 0x2f, 0x400) /usr/local/go/src/internal/poll/fd_poll_runtime.go:94 +0x3d internal/poll.(*FD).Write(0xc42545c080, 0xc42264b800, 0x2f, 0x400, 0x0, 0x0, 0x0) /usr/local/go/src/internal/poll/fd_unix.go:227 +0x244 net.(*netFD).Write(0xc42545c080, 0xc42264b800, 0x2f, 0x400, 0xc427942f20, 0x3, 0xa022617980) /usr/local/go/src/net/fd_unix.go:220 +0x52 net.(*conn).Write(0xc426176e38, 0xc42264b800, 0x2f, 0x400, 0x0, 0x0, 0x0) /usr/local/go/src/net/net.go:188 +0x6d crypto/tls.(*Conn).write(0xc422b99500, 0xc42264b800, 0x2f, 0x400, 0xc421d39ac0, 0xc427942f01, 0x613054) /usr/local/go/src/crypto/tls/conn.go:832 +0x1a7 crypto/tls.(*Conn).writeRecordLocked(0xc422b99500, 0xdad617, 0xc426fd0000, 0x12, 0x1000, 0x0, 0x0, 0x0) /usr/local/go/src/crypto/tls/conn.go:909 +0x360 crypto/tls.(*Conn).Write(0xc422b99500, 0xc426fd0000, 0x12, 0x1000, 0x0, 0x0, 0x0) /usr/local/go/src/crypto/tls/conn.go:1068 +0x1bb net/http.http2stickyErrWriter.Write(0x7f51f45266d0, 0xc422b99500, 0xc429e00b48, 0xc426fd0000, 0x12, 0x1000, 0xccc240, 0x7f51f455f8b0, 0x80000000000) /usr/local/go/src/net/http/h2_bundle.go:6803 +0x7a net/http.(*http2stickyErrWriter).Write(0xc42273b280, 0xc426fd0000, 0x12, 0x1000, 0x12, 0x412577, 0xc426150840) <autogenerated>:1 +0x74 bufio.(*Writer).Flush(0xc4241c3d80, 0x895, 0xc42cca8c00) /usr/local/go/src/bufio/bufio.go:567 +0x7e net/http.(*http2ClientConn).writeHeaders(0xc429e009c0, 0x895, 0xc42cca8c09, 0x0, 0xb6, 0x0, 0x0) /usr/local/go/src/net/http/h2_bundle.go:7436 +0x1ec net/http.(*http2ClientConn).RoundTrip(0xc429e009c0, 0xc4239b1200, 0x0, 0x0, 0x0) /usr/local/go/src/net/http/h2_bundle.go:7289 +0x46b net/http.(*http2Transport).RoundTripOpt(0xc420837220, 0xc4239b1200, 0xc424c5fb00, 0xc4241396c0, 0x60000000040b130, 0xffffffffffffffff) /usr/local/go/src/net/http/h2_bundle.go:6858 +0x145 net/http.(*http2Transport).RoundTrip(0xc420837220, 0xc4239b1200, 0x654201, 0xc424c5fb98, 0xc4241398f0) /usr/local/go/src/net/http/h2_bundle.go:6820 +0x3a net/http.http2noDialH2RoundTripper.RoundTrip(0xc420837220, 0xc4239b1200, 0xc422c7e030, 0x5, 0xc420813048) /usr/local/go/src/net/http/h2_bundle.go:990 +0x39 net/http.(*Transport).RoundTrip(0x137f9a0, 0xc4239b1200, 0x137f9a0, 0xbea9f7ed1c77e1fd, 0x9a938c0869e0) /usr/local/go/src/net/http/transport.go:371 +0xd47 net/http.send(0xc4239b1100, 0x12f72e0, 0x137f9a0, 0xbea9f7ed1c77e1fd, 0x9a938c0869e0, 0x138d2a0, 0xc42062af70, 0xbea9f7ed1c77e1fd, 0xc424139b08, 0x1) /usr/local/go/src/net/http/client.go:249 +0x1a9 net/http.(*Client).send(0xc4208f15c0, 0xc4239b1100, 0xbea9f7ed1c77e1fd, 0x9a938c0869e0, 0x138d2a0, 0xc42062af70, 0x0, 0x1, 0xbfb900) /usr/local/go/src/net/http/client.go:173 +0xfd net/http.(*Client).Do(0xc4208f15c0, 0xc4239b1100, 0x18, 0xc4200166c0, 0x18) /usr/local/go/src/net/http/client.go:602 +0x28d .......

Metadata

Metadata

Assignees

No one assigned

    Labels

    FrozenDueToAgeNeedsInvestigationSomeone must examine and confirm this is a valid issue and not a duplicate of an existing one.

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions