proposal: crypto/tls: pluggable Backend delegation to end ecosystem fragmentation

[rbqvq]

Proposal Details

Abstract

The current monolithic architecture of crypto/tls has led to significant ecosystem fragmentation, with high-performance users (e.g., Cloudflare) being forced to maintain private Go toolchain patches to implement features like Delegated Credentials.

This proposal introduces a delegation mechanism within tls.Conn to support alternative backends.

This "escape hatch" allows the standard library to remain the unified interface for Go networking while enabling industry-leading performance or feature without the need for custom toolchain forks.

Background

Go's net/http and other core libraries are hard-coupled to *tls.Conn.

This coupling, combined with the slow evolution of the standard TLS stack, has created a "Performance Gap"

Users shouldn't have to wait for Go 2 to achieve modern networking efficiency.

Proposed Changes

1. The AlternativeConn Interface

A exported interface that maps to the functional requirements of a TLS connection.

2. The alternativeConn wrapper

Adding an alt field to tls.Conn.

If set, tls.Conn acts as a transparent proxy to the backend.

type AlternativeConn interface {
    net.Conn

    ConnectionState() ConnectionState
    ... // all exists method on tls.Conn
}

type Conn struct {
    alt AlternativeConn
    // ... existing fields
}

func (c *Conn) Read(p []byte) (int, error) {
    if c.alt != nil {
        return c.alt.Read(p)
    }
    ...
}

3. Factory Method

func NewAltConn(alt AlternativeConn) *tls.Conn
func NewAltQUICConn(alt AlternativeQUICConn) *tls.QUICConn

This allows any third-party library to be used as a *tls.Conn compatible with the entire Go ecosystem.

Rationale

1. End Toolchain Fragmentation: Bring big-tech forks back to the official Go distribution.
2. Allow users to use other TLS backends (e.g. OpenSSL, BoringSSL or other backend)
3. Allow us to add experimental packages to the draft RFC (golang.org/x/crypto/tls)

Note

The proposal committee may choose to accept or reject of this proposal.

However, providing a pluggable interface is the most backward-compatible way to evolve Go's networking capabilities.

[rbqvq]

For feature add new methods:

type SomeNewMethod interface {
    Feature() error
}

func (c *Conn) Feature() error {
    if c.alt != nil {
        if alt, ok := c.alt.(SomeNewMethod); ok {
            return alt.Feature()
        }

        return errors.ErrUnsupported
    }
}

[gabyhelp]

Related Issues

• proposal: net/http: allow use of external TLS stacks #21753 (closed)
• proposal: crypto/tls: add support for delegated credentials #35311
• proposal: net/http: Upgrade http2 from not *tls.Conn #74674 (closed)
• proposal: crypto: add crypto engines #33281 (closed)
• proposal: net/http: support TLS interface for HTTP/2 #64359 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[seankhliao]

See previous issues like #33281, pluggable crypto is in general a non-goal.

[rbqvq]

See previous issues like #33281, pluggable crypto is in general a non-goal.

This solution is lighter and easier to maintain than the #33281

[rbqvq]

We can easily resolve #5465 split TLS from net/http in Go 1.

Please think it over.

[FiloSottile]

I agree the hard net/http coupling is unfortunate, and that this is lighter than #33281, but this is still too much API surface (we'd become bound by AlternativeConn, with no way to extend it without breaking the backwards compatibility promise) and generally would add a layer of indirection, which makes it harder to reason about the library.

[rbqvq]

I agree the hard net/http coupling is unfortunate, and that this is lighter than #33281, but this is still too much API surface (we'd become bound by AlternativeConn, with no way to extend it without breaking the backwards compatibility promise) and generally would add a layer of indirection, which makes it harder to reason about the library.

See #76997 (comment)

I think we just need assertions and error returns, while reserving some standard interfaces as something that must be implemented (e.g. net.Conn, io.ReaderFrom, io.WriteTo).

Also add an Alt() to return the original AltConn

This is enough for us, and we are not bound by phase compatibility, as long as we have to return error.

I think it's a good idea to give it to the proposal committee and let them discuss it.

So we don't have to wait until Go 2.

To be honest, not only net/http, but other packages such as quic-go are also hard-bundled with crypto/tls.

[rbqvq]

@FiloSottile If you think it's enough, I think it's a good idea to have the proposal committee discuss it, and actually *tls.Conn doesn't add much to the api since Go 1.x.

In addition, we can keep some of the APIs we have to implement for future consideration, as I said earlier.