net/http: server links in crypto/tls when not serving TLS

[dpinela]

What version of Go are you using (go version)?

$ go version
go version go1.12 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/dpinela/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/dpinela/dev/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.12/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.12/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/41/2xpv_r1j5n5bnflwb7s1hv580000gp/T/go-build657911368=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Build the following program:

package main

import (
	"net/http"
)

func main() {
	http.ListenAndServe("localhost:8080", nil)
}

What did you expect to see?

The resulting binary has no trace of crypto/tls in it, since it only ever serves unencrypted HTTP.

What did you see instead?

> bloaty -c bloatyconfig.txt -d gopackages hellohttp 
     VM SIZE                                          FILE SIZE
 --------------                                    --------------
  47.8%  2.71Mi runtime                             2.60Mi  37.4%
   0.0%       0 [Unmapped]                          1.40Mi  20.2%
  10.6%   615Ki [173 Others]                         607Ki   8.5%
   8.2%   474Ki net/http                             473Ki   6.6%
   7.9%   461Ki [runtime metadata]                   461Ki   6.5%
   5.9%   345Ki crypto/tls                           345Ki   4.9%
   3.6%   207Ki net                                  206Ki   2.9%
   2.0%   114Ki unicode                              114Ki   1.6%
   1.9%   111Ki reflect                              110Ki   1.6%
   1.6%  92.0Ki internal/x/text/unicode/norm        91.8Ki   1.3%
   1.5%  84.3Ki math/big                            81.8Ki   1.1%
   1.4%  82.3Ki crypto/x509                         82.2Ki   1.2%
   1.1%  66.3Ki crypto/elliptic                     66.1Ki   0.9%
   1.1%  64.4Ki time                                64.2Ki   0.9%
   1.1%  62.1Ki encoding/asn1                       62.0Ki   0.9%
   0.9%  49.4Ki internal/x/net/idna                 49.3Ki   0.7%
   0.8%  48.3Ki internal/x/crypto/chacha20poly1305  48.3Ki   0.7%
   0.8%  46.9Ki fmt                                 46.8Ki   0.7%
   0.7%  42.1Ki strconv                             42.0Ki   0.6%
   0.6%  33.3Ki strings                             33.3Ki   0.5%
   0.6%  33.0Ki syscall                             32.8Ki   0.5%
 100.0%  5.68Mi TOTAL                               6.95Mi 100.0%

Note how the TLS library and its dependencies (I'm counting math/big) take at least 685.4 KB, almost 10% of the binary's size (#6853).

(The runtime looks huge mainly because of runtime.pclntab).

bloatyconfig.txt:

custom_data_source: {
	name: "gopackages"
	base_data_source: "fullsymbols"
	rewrite: {
		pattern: "^(go|type)\\."
		replacement: "[runtime metadata]"
	}
	rewrite: {
		pattern: "^(.*?)\\.[^/]*$"
		replacement: "\\1"
	}
}

[bcmills]

CC @bradfitz @FiloSottile

[bradfitz]

You can learn why with:

$ go tool compile http.go

bradfitz@gdev:~$ go tool link -dumpdep http.o | grep -E "net/http.* -> .*tls" | head -10
net/http.init -> crypto/tls.init
type.net/http.Server -> type.*crypto/tls.Config
type.net/http.Server -> type.map[string]func(*net/http.Server, *crypto/tls.Conn, net/http.Handler)
type.map[string]func(*net/http.Server, *crypto/tls.Conn, net/http.Handler) -> type..namedata.*map[string]func(*http.Server, *tls.Conn, http.Handler)-
type.map[string]func(*net/http.Server, *crypto/tls.Conn, net/http.Handler) -> type.func(*net/http.Server, *crypto/tls.Conn, net/http.Handler)
type.map[string]func(*net/http.Server, *crypto/tls.Conn, net/http.Handler) -> type.noalg.map.bucket[string]func(*net/http.Server, *crypto/tls.Conn, net/http.Handler)
type.func(*net/http.Server, *crypto/tls.Conn, net/http.Handler) -> type..namedata.*func(*http.Server, *tls.Conn, http.Handler)-
type.func(*net/http.Server, *crypto/tls.Conn, net/http.Handler) -> type.*crypto/tls.Conn
type.noalg.map.bucket[string]func(*net/http.Server, *crypto/tls.Conn, net/http.Handler) -> type..namedata.*map.bucket[string]func(*http.Server, *tls.Conn, http.Handler)-
type.noalg.map.bucket[string]func(*net/http.Server, *crypto/tls.Conn, net/http.Handler) -> type.noalg.[8]func(*net/http.Server, *crypto/tls.Conn, net/http.Handler)

$ go tool link -dumpdep http.o | grep -E " -> .*http2" | head -10
net/http.init -> internal/x/net/http2/hpack.init
internal/x/net/http2/hpack.init -> internal/x/net/http2/hpack.initdone·
internal/x/net/http2/hpack.init -> internal/x/net/http2/hpack.init.ializers
net/http.init.ializers -> type.map[net/http.http2ErrCode]string
net/http.init.ializers -> type.map[net/http.http2FrameType]string
net/http.init.ializers -> type.map[net/http.http2FrameType]map[net/http.http2Flags]string
net/http.init.ializers -> type.map[net/http.http2Flags]string
net/http.init.ializers -> type.map[net/http.http2FrameType]net/http.http2frameParser
net/http.init.ializers -> type.map[net/http.http2SettingID]string
net/http.init.ializers -> type.net/http.http2SettingID
...

But I'm not sure this is a priority. Even if we did the games to make the linker happy enough to be able to drop things, that only helps the Server. We'd have to play the same games with the http.Transport code (as used by the http.Client) and basically you'd have to never use the http.Client or http.Transport types anywhere in your code, or ever import net/http/httputil.ReverseProxy (which uses Transport), etc, because the Transport doesn't know until runtime whether you're going to use https or not.

Actually, considering the line above:

type.net/http.Server -> type.*crypto/tls.Config

This is basically impossible to fix.

Sorry.

It's also been like this since day 1.

[bradfitz]

I'm going to close this, as I don't see a way forward, and I also don't see why it matters in a world that's increasingly using (and often only using) https.