Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

crypto/tls: TLS 1.3 client fails to connect to some servers #41910

Open
carlmjohnson opened this issue Oct 10, 2020 · 5 comments
Open

crypto/tls: TLS 1.3 client fails to connect to some servers #41910

carlmjohnson opened this issue Oct 10, 2020 · 5 comments
Milestone

Comments

@carlmjohnson
Copy link
Contributor

@carlmjohnson carlmjohnson commented Oct 10, 2020

What version of Go are you using (go version)?

go version go1.15.2 darwin/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env

GO111MODULE="on"
GOARCH="amd64"
GOBIN="/Users/adhoc/bin"
GOCACHE="/Users/adhoc/Library/Caches/go-build"
GOENV="/Users/adhoc/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/adhoc/src/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/adhoc/src/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.15.2/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.15.2/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/adhoc/src/spl/almanack/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/p7/jc4qc9n94r3f6ylg0ssh1rq00000gs/T/go-build149709333=/tmp/go-build -gno-record-gcc-switches -fno-common"

But it also happens on an AWS Lambda.

What did you do?

Attempted to get a URL from a certain server with net/http.

Here is an example URL: https://www.inquirer.com/resizer/LUs6Sqe2nxqwxVEe0NHgiLSh6i0=/arc-anglerfish-arc2-prod-pmn/public/GG5L3ZCM5JCOTDVPYIURDF3HHE.jpg

What did you expect to see?

The URL should load an image.

Interestingly, Slack fails to preview this URL, so I think that means they're using Go internally. The image should load in your browser.

What did you see instead?

The server sends an error message to net/http, but not to browsers, curl, or fasthttp.

It's unclear if Go is doing something weird and the server doesn't like it or vice versa. The server appears to be an Akamai CDN. (It's controlled by a friendly third party. I can ask for more details on Monday.) It's possible they've decided that Go is evil and are blocking it somehow, but changing the User-Agent does not help, so however they detect Go, it isn't in the obvious manner of banning a User-Agent or IP.

Here's a reproducer:

package main

import (
	"fmt"
	"io/ioutil"
	"log"
	"net/http"

	"github.com/valyala/fasthttp"
)

func main() {
	const exampleURL = "https://www.inquirer.com/resizer/LUs6Sqe2nxqwxVEe0NHgiLSh6i0=/arc-anglerfish-arc2-prod-pmn/public/GG5L3ZCM5JCOTDVPYIURDF3HHE.jpg"

	// Fails with net/http
	rsp, err := http.Get(exampleURL)
	check(err)
	blob, err := ioutil.ReadAll(rsp.Body)
	check(err)
	rsp.Body.Close()
	printBlob(blob)

	// Succeeds with fasthttp
	_, blob, err = fasthttp.Get(nil, exampleURL)
	check(err)
	printBlob(blob)
}

func check(err error) {
	if err != nil {
		log.Fatal(err)
	}
}

func printBlob(blob []byte) {
	allASCII := true
	for _, c := range blob {
		if c > 127 {
			allASCII = false
			break
		}
	}
	if allASCII {
		fmt.Printf("got: %q\n", blob)
	} else {
		fmt.Printf("got: %0x", blob[:80])
	}
}

Output:

got: "<!DOCTYPE html> <html> <body>  <h1>Invalid Site</h1>  </body> </html>"
got: ffd8ffe000104a46494600010100000100010000ffe20c584943435f50524f46494c4500010100000c484c696e6f021000006d6e74725247422058595a2007ce00020009000600310000616373704d53

In other words, fasthttp gets the JPEG. Curl does as well.

@carlmjohnson
Copy link
Contributor Author

@carlmjohnson carlmjohnson commented Oct 12, 2020

Minor update: This reproduces with the www.inquirer.com homepage, so it's not route specific.

@carlmjohnson
Copy link
Contributor Author

@carlmjohnson carlmjohnson commented Oct 12, 2020

Double update: Setting tls.Config{ MaxVersion: tls.VersionTLS12 } fixes the issue, so something is going wrong in TLS1.3 negotiation!

@bradfitz bradfitz changed the title net/http: failure to communicate to server that connects with browsers, curl, and fasthttp crypto/tls: TLS 1.3 client fails to connect to some servers Oct 12, 2020
@bradfitz bradfitz added this to the Go1.16 milestone Oct 12, 2020
@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Oct 12, 2020

/cc @FiloSottile

carlmjohnson added a commit to spotlightpa/almanack that referenced this issue Oct 12, 2020
carlmjohnson added a commit to spotlightpa/almanack that referenced this issue Oct 12, 2020
@carlmjohnson
Copy link
Contributor Author

@carlmjohnson carlmjohnson commented Oct 12, 2020

Curl appears to connect properly with TLS1.3 enabled by OpenSSL.


* Connected to www.inquirer.com (172.232.7.82) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /usr/local/etc/openssl@1.1/cert.pem
  CApath: /usr/local/etc/openssl@1.1/certs
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [29 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [4135 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [264 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [52 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [52 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: OU=Domain Control Validated; CN=www.inquirer.com
*  start date: May 15 18:56:42 2019 GMT
*  expire date: May 15 18:56:42 2021 GMT
*  subjectAltName: host "www.inquirer.com" matched cert's "www.inquirer.com"
*  issuer: C=US; ST=Arizona; L=Scottsdale; O=GoDaddy.com, Inc.; OU=http://certs.godaddy.com/repository/; CN=Go Daddy Secure Certificate Authority - G2
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
} [5 bytes data]
* Using Stream ID: 1 (easy handle 0x7f849e00a200)
} [5 bytes data]
> GET / HTTP/2
> Host: www.inquirer.com
> user-agent: curl/7.72.0
> accept: */*
> 
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [249 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [249 bytes data]
* old SSL session ID is stale, removing
{ [5 bytes data]
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
} [5 bytes data]
< HTTP/2 200 
< content-type: text/html;charset=UTF-8
< server: openresty
< x-akamai-transformed: 9 - 0 pmb=mRUM,2
< cache-control: private, max-age=60
< expires: Mon, 12 Oct 2020 15:45:50 GMT
< date: Mon, 12 Oct 2020 15:44:50 GMT
< set-cookie: arc-country=US; expires=Mon, 12-Oct-2020 15:59:50 GMT; path=/
< set-cookie: arc-zipcode=21201-21203+21205-21206+21209-21218+21223-21224+21229-21231+21233+21235+21239-21241+21250-21252+21263-21264+21270+21273+21275+21278-21282+21284-21285+21287-21290+21297-21298; expires=Mon, 12-Oct-2020 15:59:50 GMT; path=/
< server-timing: cdn-cache; desc=HIT
< server-timing: edge; dur=1
< content-security-policy: upgrade-insecure-requests
< arc-zipcode: 21201-21203+21205-21206+21209-21218+21223-21224+21229-21231+21233+21235+21239-21241+21250-21252+21263-21264+21270+21273+21275+21278-21282+21284-21285+21287-21290+21297-21298
< arc-country: US
< 
{ [2248 bytes data]
(rest omitted)
@carlmjohnson
Copy link
Contributor Author

@carlmjohnson carlmjohnson commented Oct 16, 2020

Is it possible this is related to #40521?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.