import/path: Slow compilation in a sandboxed environment in the absence of a shared build cache location

[ysabih2]

Go version

go version go1.21.4 linux/amd64

What operating system and processor architecture are you using (go env)?

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/var/tmp/go-build'
GOENV='/root/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/root/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/root/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/lib/go-1.21'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/lib/go-1.21/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.21.4'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build3811157868=/tmp/go-build -gno-record-gcc-switches'

What did you do?

I'm working on a service that runs user-submitted code which consists of a solution and tests. The service runs in a Docker container based on Ubuntu 20.04. The code is compiled and executed in a sandboxed environment, the sandboxing is performed using https://github.com/ioi/isolate. I've noticed that compilation is significantly slower when there is no shared build cache directory. As a workaround, I've changed the cache location to /var/tmp/go-build, and I run go build std during the image build to populate the cache. The cache directory is accessible as read-only to the sandboxes.

What did you expect to see?

I expected compilation time not to be dependent on a shared cache directory because the user-submitted code only references the standard library. I use other compilers for C and C# in sandboxed environment and their performance is acceptable.

What did you see instead?

Compilation is very slow, it can take up to 20s for small files depending on the host VM in the absence of a shared build cache.

[seankhliao]

you can precompile the standard library and package that if you wish (the go distribution used to do that).
I don't think there's anything concrete to do here.

[ysabih2]

you can precompile the standard library and package that if you wish (the go distribution used to do that).
I don't think there's anything concrete to do here.

@seankhliao Thanks for the quick reply! Is there some documentation on how to do that? I've found some answers online but they seem to be outdated and didn't work for me.