cmd/compile: slow compilation on generated code

[mschoch]

What version of Go are you using (go version)?

$ go version
go version go1.7 darwin/amd64

What operating system and processor architecture are you using (go env)?

$ go env
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/mschoch/go"
GORACE=""
GOROOT="/Users/mschoch/Documents/research/gosrc"
GOTOOLDIR="/Users/mschoch/Documents/research/gosrc/pkg/tool/darwin_amd64"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"

What did you do?

$ mkdir /tmp/gp
$ export GOPATH=/tmp/gp
$ go get -tags 'prod' github.com/blevesearch/segment

What did you expect to see?

Under 1.6 this runs for several seconds, but compiles successfully in somewhat reasonable time. It was always slow (due to very large generated source files which is why we protect this version behind the 'prod' build tag).

What did you see instead?

At this point it appears to hang for minutes. Running top shows:

PID    COMMAND      %CPU TIME     #TH   #WQ  #PORT MEM    PURG   CMPRS  PGRP  PPID  STATE    BOOSTS         %CPU_ME %CPU_OTHRS UID  FAULTS    COW    MSGSENT   MSGRECV   SYSBSD    SYSMACH   CSW        PAGEINS IDLEW    POWE USER
48037  compile      96.7 18:16.05 5/1   0    17    623M   0B     0B     48033 48033 running  *0[1]          0.00000 0.00000    501  160783    23     78        6         109603+   568       2165690+   10      129      96.7 mschoc

I have given it at least 10 minutes now.

Additional Info

This repo contains Go source generated by the Ragel state machine compiler. The file in question segment_words_prod.go is 2.57 MB. Direct link to this file: https://raw.githubusercontent.com/blevesearch/segment/master/segment_words_prod.go
This particular version was generated with the -G2 flags which. The version produced for use without the prod build tag uses-F1 continues to work fine in 1.7.

[bradfitz]

I believe this is a dup of another SSA regression or so, but I forget which ones.

Can you try Go tip?

/cc @dr2chase @randall77

[mschoch]

I tried again with tip:

$ go version
go version devel +22d3bf1 Thu Sep 15 19:24:04 2016 +0000 darwin/amd64

Still compiling after 10 minutes.

Looks like it may be duplicate of #16407

[bradfitz]

Yeah, let's merge this into #16407. Please comment on that bug with a link to your code.

[randall77]

Probably not a dup of #16407 after all. Reopening.

[randall77]

After CL 30163 (new phi building algorithm), this example compiles for me in ~8 minutes. Most of the time (~7 min) is lowered CSE.

As an experiment, I changed CSE's cmpDepth from 4 to 1 and it compiled in under 1 minute.
@tzneal , was there a particular reason cmpDepth is 4? My uninformed opinion is that it seems high.

[tzneal]

IIRC, it was a good value given some benchmarks I was using. So much has changed since then though, that it may not be a good choice anymore.

[gopherbot]

CL https://golang.org/cl/30257 mentions this issue.