Build-time source modification using -toolexec

[Julio-Guerra]

What version of Go are you using (go version)?

$ go version
go version go1.13.1 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/julio/.cache/go-build"
GOENV="/home/julio/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/julio/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go-1.13"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.13/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build292859470=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I am trying to create a tool to do build-time instrumentation of a Go program using the go build option -toolexec such as go build -toolexec my-tool .
It intercepts compiler calls and generates new Go files in the temporary build directory before replacing them in the compiler call.

It was working fine until I added an import to a new package.

What did you expect to see?

Being able to do add new imports that would be resolved at link-time.

What did you see instead?

But since this is happening after go build has done his dependency listing, the new import is not taken into account and the compilation fails because the new import is not in the importcfg file nor even compiled actually.

I tried to add it myself with a pre-compiled package but it is still detected by a compiler internal check:

/tmp/go-build445887355/b040/binary.go:29:2: internal compiler error: conflicting package heights 20 and 6 for path "reflect"

Note that since I want to instrument everything, I use the -a option to also instrument the standard library.

[ianlancetaylor]

As you've discovered, the -toolexec option is not powerful enough to support arbitrary source code rewriting. This is not a bug, it's just the way the option works. We are not going to add this capability, which would be difficult to implement safely. Sorry.

[Julio-Guerra]

For the record, I did use the option -toolexec in the end in order to perform compile-time instrumentation. The limitations were avoided using the Go directive linkname to "export" or "import" function definitions from one package to another.