Skip to content

net/http: All outgoing http requests are canceled if 1 dial times out #24194

Closed
@jsaltermedialab

Description

@jsaltermedialab

What version of Go are you using (go version)?

go version go1.9.4 darwin/amd64
also go 1.8

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/jasonsalter/go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/0v/c3_7c31s20j_l96jbslwg9gh0000gn/T/go-build630477543=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"

What did you do?

package main

import (
  "fmt"
  "net/http"
  "net"
  "time"
  "context"
)

func main() {
  // channel to not exit main before requests finish
  d := make(chan bool)

  // make a RoundTripper and client so we dont have to use the default objects.
  // gurantees that requests made with this client trigger a dial error
  var DefaultOneMobileTransport http.RoundTripper = &http.Transport{
    Proxy: http.ProxyFromEnvironment,
    DialContext: (&net.Dialer{
      Timeout:   1 * time.Millisecond,
    }).DialContext,
  }

  badClient := &http.Client{
    Transport: DefaultOneMobileTransport,
  }

  // make a request and use a context from one of the 2 options in context.go
  r, _ :=   http.NewRequest("GET", "https://httpbin.org/delay/1", nil)
  c,_ := context.WithCancel(context.TODO())
  r = r.WithContext(c)

  // increasing 1 to higher numbers makes the error more repeatable
  for range(make([]int, 1)) {
    go func() {
      // request will have a dial error basically immediately
      _, err := badClient.Do(r)
      fmt.Printf("%s\n", err)
    }()
  }

  // start background request with default client/transports
  go func() {
    // make a request and try the other context
    // this endpoint waits at least 1 second and then returns
    // but the dial error from the other client kills it even though
    // it shouldnt share any dependencies
    r, _ :=   http.NewRequest("GET", "https://httpbin.org/delay/1", nil)
    c,_ := context.WithCancel(context.Background())
    r = r.WithContext(c)

    _, err := http.DefaultClient.Do(r)
    // should print nil but gets canceled by another bad request
    fmt.Printf("\ndefault %s\n", err)
    d <- true
  }()
  <- d
}

What did you expect to see?

The program uses the default http client and transport and a completely separate client/transport but the requests made from both share the same context so when one request fails all requests fail. Even if you remove the context additions to the requests they will use the same defaults and the error still happens. I only added in the calls because every context is either a child of todo or background and was trying to get it to work. The requests should not affect each other since they are made from different clients.

What did you see instead?

A request gets canceled because of another request from a different client getting a dial error.

I am making a program that makes a LOT of short lived requests and noticed that this was happening so I read through the source code some and can't seem to find a way to make the requests in the example not dependent in any way. I think the root of the problem is that the contexts are shared between requests.

If someone could look at this or provide any feedback on things to try I will gladly do whatever is necessary to help. Thanks.

Metadata

Metadata

Assignees

No one assigned

    Labels

    FrozenDueToAgeWaitingForInfoIssue is not actionable because of missing required information, which needs to be provided.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions