Construct a utls.Config from a tls.Config

[AxbB36]

At #16 (comment) I had to adapt utls to the interface of http2.Transport. The DialTLS callback receives a golang net/crypto tls.Config. To create a UClient I need a utls.Config. I can't just pass on the tls.Config, because the two types are incompatible.

I manually created a new utls.Config and copied the NextProtos member because it was important for my example, but in general there are other important fields that may need to be copied.

uconn := utls.UClient(conn, &utls.Config{NextProtos: cfg.NextProtos}, *utlsClientHelloID)

Mainline crypto/tls provides a Config.Clone method that internally knows which fields are safe to copy (not all are safe to copy, for example locks). Does it make sense to have something similar to create a utls.Config from an existing tls.Config?

[sergeyfrolov]

utls.Config and tls.Config are same structs, aside for a type.

create a utls.Config from an existing tls.Config

If you believe that it is generally useful , I wouldn't object to a PR that generates tls.Config from utls.Config by doing a deep copy. However, that code doesn't really have to be in the uTLS: user has access to all the fields of both Configs.

[AxbB36]

It's not as easy as it seems at first, because

• The version of crypto/tls may not match the version of utls, so their Config structs may not have the same fields.
• Not only the types of tls.Config and utls.Config are incompatible, but also some of their fields. For example, if you try to assign utlsConfig.Certificates = tlsConfig.Certificates, you get an error because each is defined in terms of its own Certificate type:

  cannot use tlsConfig.Certificates (type []"crypto/tls".Certificate) as type []"github.com/refraction-networking/utls".Certificate in assignment
  

For me, a perfect 1:1 clone isn't necessary. I can just try not to use the fields that have problems. I worked out a demo using a reflection trick that copies public fields that have a compatible type.

Click to expand program

package main

import (
	"crypto/tls"
	"fmt"
	"net"
	"net/http"
	"os"
	"reflect"
	"strings"

	utls "github.com/refraction-networking/utls"
	"golang.org/x/net/http2"
)

// For each field in dst, copy the corresponding named fields from src that are
// setting and have the same type. The idea of copying structs via reflection
// comes from a post by Nick Craig-Wood:
// https://groups.google.com/d/msg/Golang-Nuts/SDiGYNVE8iY/89hRKTF4BAAJ
func copyPublicFields(dst, src interface{}) {
	dstValue := reflect.ValueOf(dst).Elem()
	dstType := dstValue.Type()
	srcValue := reflect.ValueOf(src).Elem()
	srcType := srcValue.Type()
	for i := 0; i < dstValue.NumField(); i++ {
		dstField := dstValue.Field(i)
		name := dstType.Field(i).Name

		if !dstField.CanSet() {
			fmt.Printf("%2v cannot set %v\n", i, name)
			continue
		}

		dstTypeField := dstType.Field(i)
		srcTypeField, ok := srcType.FieldByName(name)
		if !ok {
			fmt.Printf("%2v src missing field %q\n", i, name)
			continue
		}

		if dstTypeField.Type != srcTypeField.Type {
			fmt.Printf("%2v different types for field %q\n", i, name)
			continue
		}

		value := srcValue.FieldByName(name)
		fmt.Printf("%2v Set %v = %v\n", i, name, value)
		dstField.Set(value)
	}
}

func utlsConfigFromTLSConfig(tlsConfig *tls.Config) *utls.Config {
	utlsConfig := new(utls.Config)
	copyPublicFields(utlsConfig, tlsConfig)
	return utlsConfig
}

func dialTLS(network, addr string, tlsConfig *tls.Config) (net.Conn, error) {
	fmt.Printf("DialTLS %v %v %v\n", network, addr, tlsConfig)

	colonPos := strings.LastIndex(addr, ":")
	if colonPos == -1 {
		colonPos = len(addr)
	}
	tlsConfig.ServerName = addr[:colonPos]

	conn, err := net.Dial(network, addr)
	if err != nil {
		return nil, err
	}
	// Convert the *tls.Config to a *utls.Config for UClient.
	utlsConfig := utlsConfigFromTLSConfig(tlsConfig)
	uconn := utls.UClient(conn, utlsConfig, utls.HelloFirefox_Auto)
	err = uconn.Handshake()
	return uconn, err
}

func main() {
	// Use http2.Transport.DialTLS as an example of where the stdlib may
	// give us a *tls.Config that we have to convert into a *utls.Config.
	rt := http2.Transport{
		DialTLS: dialTLS,
	}
	req, err := http.NewRequest("GET", "https://golang.org/robots.txt", nil)
	if err != nil {
		fmt.Fprintf(os.Stderr, "error: %v\n", err)
		os.Exit(1)
	}
	resp, err := rt.RoundTrip(req)
	if err != nil {
		fmt.Fprintf(os.Stderr, "error: %v\n", err)
		os.Exit(1)
	}
	fmt.Printf("%v\n", resp.Status)
}

The output of this program shows which fields are copied and which are not. This is with go1.10.5:

DialTLS tcp golang.org:443 &{<nil> <nil> [] map[] <nil> <nil> <nil> <nil> <nil> [h2] golang.org 0 <nil> false [] false false [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] <nil> 0 0 [] false 0 <nil> {{0 0} 0} {{0 0} 0 0 0 0} []}
 0 Set Rand = <nil>
 1 Set Time = <nil>
 2 different types for field "Certificates"
 3 different types for field "NameToCertificate"
 4 different types for field "GetCertificate"
 5 different types for field "GetClientCertificate"
 6 different types for field "GetConfigForClient"
 7 Set VerifyPeerCertificate = <nil>
 8 Set RootCAs = <nil>
 9 Set NextProtos = [h2]
10 Set ServerName = golang.org
11 different types for field "ClientAuth"
12 Set ClientCAs = <nil>
13 Set InsecureSkipVerify = false
14 Set CipherSuites = []
15 Set PreferServerCipherSuites = false
16 Set SessionTicketsDisabled = false
17 Set SessionTicketKey = [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
18 different types for field "ClientSessionCache"
19 Set MinVersion = 0
20 Set MaxVersion = 0
21 different types for field "CurvePreferences"
22 Set DynamicRecordSizingDisabled = false
23 different types for field "Renegotiation"
24 Set KeyLogWriter = <nil>
25 cannot set serverInitOnce
26 cannot set mutex
27 cannot set sessionTicketKeys

It works for the fields that are important to me, which are VerifyPeerCertificate, ServerName, InsecureSkipVerify, so I'm planning to try out this technique.

Are there any negative side effects of setting fields in a utls.Config? I assume that many of them (NextProtos, CipherSuites, CurvePreferences) will be overridden by uTLS anyway.

[sergeyfrolov]

Are there any negative side effects of setting fields in a utls.Config? I assume that many of them (NextProtos, CipherSuites, CurvePreferences) will be overridden by uTLS anyway.

Yes, some (but not all) of them will be overwritten. If desired field will be overwritten, you can force uTLS to fill the config first by calling BuildHandshakeState(), and then overwrite it yourself.

@AxbB36 would it be correct to say that you would have no need to construct utls.Config out of tls.Config, if #16 (comment) is implemented?

[AxbB36]

I don't really have a need for this anymore. In the http2 DialTLS callback, I'm ignoring the tls.Config and substituting a static utls.Config, which isn't the best, but it's good enough for this case.