net/http: HTTPS requests when the server does not support it -> record overflow

[eidge]

Making an HTTPS request against a server that has no TLS enabled returns an 'record overflow' error.

I wonder if the message couldn't be a bit more helpful, like: 'No valid TLS certificate found. Aborting request.'

The sample program below illustrates the problem.

package main

import "net"
import "net/http"
import "net/http/httptest"
import "fmt"

func main() {
    server := httptest.NewUnstartedServer(
        http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            fmt.Fprintf(w, "it works")
        }))

    server.Listener, _ = net.Listen("tcp", ":8080")
    server.Start()
    defer server.Close()

    req, err := http.NewRequest("GET", "https://localhost:8080", nil)
    if err != nil {
        panic(err.Error())
    }
    _, err = http.DefaultClient.Do(req)
    if err != nil {
        panic(err.Error()) // Will throw 'record overflow' here, works fine if one requests http instead of https though.
    }
}

[minux]

The crypto/tls package will try to parse the response "HTTP/1.1 400 Bad Request\r\n\r\n" as a TLS record, which it failed. I don't thinking we can do any better. crypto/tls is not HTTPS specific, so it doesn't make sense to detect this case specifically.

[bradfitz]

I mean, we could do something here and record the first few bytes before giving it to crypto/tls. And then when crypto/tls gives us an error, we could look at the bytes we copied and see if it looks like "HTTP/*" and return a better error.

Pretty low priority, but a fine feature request.

[minux]

OK, I think it's better to make crypto/tls.(*Client).Handshake return a the server response if the very first response doesn't parse correctly; rather than cache the first few bytes in net/http (as it will involve another layer of wrapping on net.Conn.) And this seems like a general purpose enhancement for crypto/tls. Actually, I think just make crypto/tls return the first few bytes of response in error message is enough, we don't need to change net/http. For example, if the example panics with: tls: oversized record received with length 20527 (server response: "HTTP/1.1 400 Bad Request"...) Then it's much better than the current message and can help the user pinpoint the problem very quickly.

[eidge]

Yes that would be a lot more helpful.

I lost a good half-an-hour trying to pin point the problem. @minux suggestion would send me immediately in the right direction.

Thanks

[bradfitz]

Turns out this is pretty easy to propagate. crypto/tls returns an untyped fmt.Errorf error, so we can make it a real type with the data. The data is easily accessible.

diff --git a/src/crypto/tls/conn.go b/src/crypto/tls/conn.go 
index e3dcf15..2a41beb 100644
--- a/src/crypto/tls/conn.go
+++ b/src/crypto/tls/conn.go
@@ -568,7 +568,7 @@ Again:
        }
        if n > maxCiphertext { 
                c.sendAlert(alertRecordOverflow) 
-               return c.in.setErrorLocked(fmt.Errorf("tls: oversized record received with length %d", n)) 
+               return c.in.setErrorLocked(fmt.Errorf("tls: oversized record received with length %d ; data=%q", n, b.data)) 
        }
        if !c.haveVers {
                // First message, be extra suspicious: this might not be a TLS

Makes the demo program above error out with the "HTTP/1.1 400 Bad Request\r\nContent-Type: text/plain\r\nConnection: close\r\n\r\n400 Bad Request"

[gopherbot]

CL https://golang.org/cl/16075 mentions this issue.

[gopherbot]

CL https://golang.org/cl/16078 mentions this issue.

[gopherbot]

CL https://golang.org/cl/16079 mentions this issue.