Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: Request to certain server hangs, works fine with curl #21301

Closed
wjdp opened this issue Aug 3, 2017 · 6 comments

Comments

Projects
None yet
4 participants
@wjdp
Copy link

commented Aug 3, 2017

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.8.3 linux/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/will/go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/will/go-build795220856=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

What did you do?

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.

package main

import (
	"fmt"
	"net/http"
	"time"
)

func main() {
	client := &http.Client{
		Timeout: 30 * time.Second,
	}
	resp, err := client.Get("https://www.southampton.ac.uk/")
	if err != nil {
		panic(err)
	}
	fmt.Println(resp)
}

What did you expect to see?

A response within a second or two

What did you see instead?

Timeout after 30 seconds, will happily hang for longer if timeout extended.

The following is the output of curl on the same machine / network. This timeout issue with this server has also been observed on a Travis test runner.

~ ❯ curl -v https://www.southampton.ac.uk/
*   Trying 152.78.118.52...
* Connected to www.southampton.ac.uk (152.78.118.52) port 443 (#0)
* found 173 certificates in /etc/ssl/certs/ca-certificates.crt
* found 697 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_256_GCM_SHA384
* 	 server certificate verification OK
* 	 server certificate status verification SKIPPED
* 	 common name: www.southampton.ac.uk (matched)
* 	 server certificate expiration date OK
* 	 server certificate activation date OK
* 	 certificate public key: RSA
* 	 certificate version: #3
* 	 subject: OU=Domain Control Validated,CN=www.southampton.ac.uk
* 	 start date: Tue, 13 Jan 2015 00:00:00 GMT
* 	 expire date: Fri, 12 Jan 2018 23:59:59 GMT
* 	 issuer: C=NL,ST=Noord-Holland,L=Amsterdam,O=TERENA,CN=TERENA SSL CA 2
* 	 compression: NULL
* ALPN, server did not agree to a protocol
> GET / HTTP/1.1
> Host: www.southampton.ac.uk
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: nginx
< Date: Thu, 03 Aug 2017 21:38:44 GMT
< Content-Type: text/html;charset=UTF-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Set-Cookie: JSESSIONID=9E437837C159A9276F8DA64CFF182DE7; Path=/; HttpOnly
< X-XSS-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
< X-Content-Type-Options: nosniff
< Referrer-Policy: unsafe-url
< Content-Security-Policy-Report-Only: default-src 'self' 'unsafe-inline' *.atbar.org *.issuu.com ddfbm2kt5ml6l.cloudfront.net cdn.soton.ac.uk cdn.southampton.ac.uk corporate.wordpress.soton.ac.uk blog.soton.ac.uk; script-src 'self' 'unsafe-inline' 'unsafe-eval' code.jquery.com www.googleadservices.com www.googletagmanager.com *.google-analytics.com connect.facebook.net static.ads-twitter.com j.flxpxl.com hj.flxpxl.com d2hlpp31teaww3.cloudfront.net d1d7fjtb6d4i2m.cloudfront.net analytics.twitter.com ib.adnxs.com www.intelliworkschat.com access.ecs.soton.ac.uk *.atbar.org www.youtube.com s.ytimg.com *.issuu.com ddfbm2kt5ml6l.cloudfront.net cdn.soton.ac.uk cdn.southampton.ac.uk stats.wp.com soton.disqus.com stats.wp.com corporate.wordpress.soton.ac.uk blog.soton.ac.uk s0.wp.com use.typekit.net connect.facebook.com platform.twitter.com; font-src 'self' data: ddfbm2kt5ml6l.cloudfront.net cdn.soton.ac.uk cdn.southampton.ac.uk use.typekit.net; img-src data: 'self' www.southampton.ac.uk cdn.soton.ac.uk cdn.southampton.ac.uk t.co mpp2.vindicosuite.com *.google-analytics.com *.facebook.com pixel.adsafeprotected.com *.doubleclick.net www.google.com www.google.co.uk *.atbar.org *.issuu.com image.isu.pub www.intelliworkschat.com pixel.wp.com blog.soton.ac.uk platform.twitter.com pbs.twimg.com syndication.twitter.com p.typekit.net d1d7fjtb6d4i2m.cloudfront.net; frame-src 'self' www.google.com www.google.co.uk www.youtube.com *.issuu.com ddfbm2kt5ml6l.cloudfront.net cdn.soton.ac.uk cdn.southampton.ac.uk; child-src 'self' www.google.com www.google.co.uk www.youtube.com *.issuu.com ddfbm2kt5ml6l.cloudfront.net cdn.soton.ac.uk cdn.southampton.ac.uk;
< Set-Cookie: BIGipServer~CorpWeb~pool-corpweb-prod-80=2678630592.20480.0000; path=/; Httponly; Secure
< 
<!DOCTYPE html>
@bradfitz

This comment has been minimized.

Copy link
Member

commented Aug 3, 2017

This is a dup of #20979

The server negotiates "h2" and has "BIGipServer" in its response headers.

I can verify with GODEBUG=http2debug that it does the immediate GOAWAY thing. Closing this bug and moving to #20979

@wjdp

This comment has been minimized.

Copy link
Author

commented Aug 3, 2017

Apologies for the duplicate and thanks for the quick response 😄

@odeke-em

This comment has been minimized.

Copy link
Member

commented Aug 3, 2017

Oops, nvm @bradfitz beat me to it. @wjdp if you want to use http1, please use

package main

import (
	"crypto/tls"
	"flag"
	"fmt"
	"net/http"
	"time"
)

func main() {
	var http1 bool
	flag.BoolVar(&http1, "http1", false, "use http1 to invoke the server")
	flag.Parse()

	tr := &http.Transport{}
	if http1 {
		tr.TLSClientConfig = &tls.Config{
			NextProtos: []string{"h1"},
		}
	}

	client := &http.Client{
		Timeout:   30 * time.Second,
		Transport: tr,
	}
	resp, err := client.Get("https://www.southampton.ac.uk/")
	if err != nil {
		panic(err)
	}
	fmt.Println(resp)
}

and invoke it with flag http1 ie go run main.go --http1 and that should return your result ASAP.

@wjdp

This comment has been minimized.

Copy link
Author

commented Aug 3, 2017

Thanks @odeke-em, provided snippet does the trick!

@mikioh mikioh changed the title net/http Request to certain server hangs, works fine with curl net/http: Request to certain server hangs, works fine with curl Aug 3, 2017

@HaraldNordgren

This comment has been minimized.

Copy link
Member

commented Jul 26, 2018

I had a problem with a hanging host as well. On the surface very similar to this case although setting the protocol version did not work.

For me it was necessary in the TLS config to use

	DynamicRecordSizingDisabled: true

I posted a more fleshed out comment about it in: #20979 (comment) for debugging similar issues.

@bradfitz

This comment has been minimized.

Copy link
Member

commented Jul 26, 2018

@HaraldNordgren, please file a separate crypto/tls bug for your case. It sounds like a crypto/tls issue instead of a net/http issue.

FalseChord pushed a commit to FalseChord/readr-restful that referenced this issue Oct 12, 2018

Yen
Solve problem that sending "get" http request to ettoday will cause t…
…imeout.

1. Adcording to golang/go#21301, we use http1 in transport layer to prevent timeout.
2. Set meta parser timeout to 5 second, this prevent API blocking.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.