x/net/http2: cannot create 'MaxConcurrentStreams' streams with a single ClientConn

[witriew]

What version of Go are you using (go version)?

$ go version
go version go1.13.3 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/wit/Library/Caches/go-build"
GOENV="/Users/wit/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/wit/gocode"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.13.3/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.13.3/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/mq/qzdh1j2n5fg3kylx9v7n79p40000gn/T/go-build653171154=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Attempt to create max concurrent streams with one client connection.

package main

import (
  "crypto/tls"
  "fmt"
  "net"
  "net/http"
  "net/http/httptest"
  "strings"
  "sync"
  "time"

  "golang.org/x/net/http2"
  "golang.org/x/net/http2/h2c"
)

const maxConcurrentStreams = 10

func handlerFunc(delayChan chan struct{}) http.HandlerFunc {
  return func(rw http.ResponseWriter, req *http.Request) {
    defer req.Body.Close()

    if delayChan != nil {
      <-delayChan
    }

    rw.WriteHeader(204)
    rw.Write([]byte("ok"))
  }
}

func main() {
  delayChan := make(chan struct{})

  h2cServer := httptest.NewServer(
    h2c.NewHandler(http.HandlerFunc(handlerFunc(delayChan)),
      &http2.Server{
        MaxConcurrentStreams: maxConcurrentStreams,
      },
    ),
  )
  defer h2cServer.Close()

  req, err := http.NewRequest("GET", "http://"+h2cServer.Listener.Addr().String(), strings.NewReader(""))
  if err != nil {
    panic(err)
  }

  h2Transport := &http2.Transport{
    AllowHTTP: true,
    DialTLS: func(netw, addr string, cfg *tls.Config) (net.Conn, error) {
      return net.Dial(netw, addr)
    },
  }

  conn, err := net.Dial("tcp", h2cServer.Listener.Addr().String())
  if err != nil {
    panic(err)
  }

  cc, err := h2Transport.NewClientConn(conn)
  if err != nil {
    panic(err)
  }

  var wg sync.WaitGroup

  wg.Add(1)
  go func() {
    defer wg.Done()

    wg.Add(maxConcurrentStreams-1)
    for i := 0; i < maxConcurrentStreams-1; i++ {
      go func(i int) {
        defer wg.Done()

        resp, err := cc.RoundTrip(req)
        fmt.Println("request:", i, resp, err)
      }(i)
    }
  }()

  time.Sleep(50 * time.Millisecond)

  wg.Add(1)
  go func() {
    defer wg.Done()

    resp, err := cc.RoundTrip(req)
    fmt.Println("max streams request:", resp, err)
  }()

  close(delayChan)
  wg.Wait()
}

What did you expect to see?

Expected to see the last RoundTrip request succeed.

What did you see instead?

$ go run go_http2_max_streams.go
max streams request: <nil> http2: client conn not usable
request: 6 &{204 No Content 204 HTTP/2.0 2 0 map[Date:[Wed, 27 Nov 2019 00:08:17 GMT]] {0xc00008ef30} 0 [] false false map[] 0xc0000da000 <nil>} <nil>
request: 8 &{204 No Content 204 HTTP/2.0 2 0 map[Date:[Wed, 27 Nov 2019 00:08:17 GMT]] {0xc00008ef30} 0 [] false false map[] 0xc0000da000 <nil>} <nil>
request: 1 &{204 No Content 204 HTTP/2.0 2 0 map[Date:[Wed, 27 Nov 2019 00:08:17 GMT]] {0xc00008ef30} 0 [] false false map[] 0xc0000da000 <nil>} <nil>
request: 5 &{204 No Content 204 HTTP/2.0 2 0 map[Date:[Wed, 27 Nov 2019 00:08:17 GMT]] {0xc00008ef30} 0 [] false false map[] 0xc0000da000 <nil>} <nil>
request: 7 &{204 No Content 204 HTTP/2.0 2 0 map[Date:[Wed, 27 Nov 2019 00:08:17 GMT]] {0xc00008ef30} 0 [] false false map[] 0xc0000da000 <nil>} <nil>
request: 4 &{204 No Content 204 HTTP/2.0 2 0 map[Date:[Wed, 27 Nov 2019 00:08:17 GMT]] {0xc00008ef30} 0 [] false false map[] 0xc0000da000 <nil>} <nil>
request: 2 &{204 No Content 204 HTTP/2.0 2 0 map[Date:[Wed, 27 Nov 2019 00:08:17 GMT]] {0xc00008ef30} 0 [] false false map[] 0xc0000da000 <nil>} <nil>
request: 0 &{204 No Content 204 HTTP/2.0 2 0 map[Date:[Wed, 27 Nov 2019 00:08:17 GMT]] {0xc00008ef30} 0 [] false false map[] 0xc0000da000 <nil>} <nil>
request: 3 &{204 No Content 204 HTTP/2.0 2 0 map[Date:[Wed, 27 Nov 2019 00:08:17 GMT]] {0xc00008ef30} 0 [] false false map[] 0xc0000da000 <nil>} <nil>

The issue appears to be at https://github.com/golang/net/blob/master/http2/transport.go#L735:

    maxConcurrentOkay = int64(len(cc.streams)+1) < int64(cc.maxConcurrentStreams)

where the test is to see whether an additional stream can be handled, but the condition should be <= instead of <.

[dmitshur]

/cc @bradfitz @tombergan per owners.

[AlexanderYastrebov]

Could be fixed by https://go-review.googlesource.com/c/net/+/352469