cmd/compile: compile time takes far longer with Go 1.7.5 than three prior Go releases

[codekoala]

What version of Go are you using (go version)?

go version go1.7.5 linux/amd64

What operating system and processor architecture are you using (go env)?

Arch Linux

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/codekoala/dev/go"
GORACE=""
GOROOT="/usr/lib/go"
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build837706947=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"

What did you do?

% docker run -it --rm golang:1.4 /bin/bash -c "go version ; time go get -u github.com/trustpath/sequence/cmd/sequence"
go version go1.4.3 linux/amd64

real	0m11.227s
user	0m4.493s
sys	0m0.747s


% docker run -it --rm golang:1.5 /bin/bash -c "go version ; time go get -u github.com/trustpath/sequence/cmd/sequence"
go version go1.5.4 linux/amd64

real	0m11.812s
user	0m7.930s
sys	0m0.910s


% docker run -it --rm golang:1.6 /bin/bash -c "go version ; time go get -u github.com/trustpath/sequence/cmd/sequence"
go version go1.6.4 linux/amd64

real	0m12.258s
user	0m8.063s
sys	0m0.663s


% docker run -it --rm golang:1.7 /bin/bash -c "go version ; time go get -u github.com/trustpath/sequence/cmd/sequence"
go version go1.7.5 linux/amd64

real	1m26.425s
user	1m21.607s
sys	0m1.087s


% docker run -it --rm golang:1.8 /bin/bash -c "go version ; time go get -u github.com/trustpath/sequence/cmd/sequence"
go version go1.8rc3 linux/amd64

real	0m31.772s
user	0m30.927s
sys	0m1.117s

What did you expect to see?

I expected Go 1.7 and 1.8 to be at least as fast as previous versions or maybe a little slower.

What did you see instead?

Go 1.7.5 took significantly longer to compile than three previous versions, and Go 1.8rc3 took 3x as long as 1.4.3, 1.5.4, and 1.6.4.

[quentinmit]

If go get -v is to be believed, it was github.com/trustpath/sequence that took the bulk of the time to compile. I don't see anything particularly complicated there, in fact, the longest file is only 1,429 lines. Possibly it's the ~1,000 line switch statement in reqmethods.go? Thanks for the repro!

[josharian]

Thanks! I'll put this in my queue to investigate; on my phone now. Could definitely be the large switch, that's a known source of some problems. (Might also try tip—there's at least one optimization in there to help with similar code.)

[josharian]

The slow package is actually github.com/zhenjl/xparse/etld, so I'm going to close this as a duplicate of #16407.

[josharian]

Btw, the easiest way to see what the slow package is is to run:

go build -v -a -toolexec=time github.com/trustpath/sequence

Here's the non-stdlib part of the output from that:

github.com/BurntSushi/toml
golang.org/x/net/idna
# golang.org/x/net/idna
        0.03 real         0.02 user         0.00 sys
github.com/zhenjl/xparse/etld
# github.com/BurntSushi/toml
        0.33 real         0.32 user         0.02 sys
# encoding/json
        0.34 real         0.32 user         0.03 sys
github.com/willf/bitset
# github.com/willf/bitset
        0.08 real         0.08 user         0.01 sys
# github.com/willf/bitset
        0.00 real         0.00 user         0.00 sys
# github.com/zhenjl/xparse/etld
       33.87 real        37.17 user         1.20 sys
github.com/trustpath/sequence
# github.com/trustpath/sequence
        0.23 real         0.27 user         0.01 sys

[codekoala]

@josharian thank you for the tip! I was not aware it was possible to get that kind of information.

[josharian]

No prob. Please keep filing issues about packages that are unusually slow to compile. They're very helpful in flushing out and fixing performance bugs.