New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: slow "native" performance with Mac OS X 10.14.1 and 10.12.6 #28739

Open
myitcv opened this Issue Nov 12, 2018 · 5 comments

Comments

Projects
None yet
6 participants
@myitcv
Member

myitcv commented Nov 12, 2018

Following a debug session with @fatih

What version of Go are you using (go version)?

go version go1.11.2 darwin/amd64

# and

go version go1.11.2 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
GOARCH="amd64"
GOBIN="/Users/fatih/go/bin"
GOCACHE="/Users/fatih/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/fatih/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.11.2/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.11.2/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/k_/87syx3r50m93m72hvqj2qqlw0000gn/T/go-build130848285=/tmp/go-build -gno-record-gcc-switches -fno-common"

and

GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build416231211=/tmp/go-build -gno-record-gcc-switches"

What did you do?

The symptom that @fatih and I were investigating was godef performing ~2x more slowly on his fast machine than on my slower machine.

@fatih's machine specs for reference:

 Model Name:    iMac Pro
  Model Identifier:    iMacPro1,1
  Processor Name:    Intel Xeon W
  Processor Speed:    3 GHz
  Number of Processors:    1
  Total Number of Cores:    10
  L2 Cache (per Core):    1 MB
  L3 Cache:    13.8 MB
  Memory:    64 GB

OS X 10.14.1 (18B75)

We think we have a smaller reproduction that demonstrates what appears to be an OS X 10.14.n issue that may or may not be related to Go. But go list appears to be a good way to demonstrate the problem and hopefully therefore a good place to start further investigation.

The following was run in a Terminal on @fatih's setup, and then within a Docker container on the same machine. Run-times of the go list commands were compared:

/bin/bash
(
set -eux
export GOPATH=$(mktemp -d)
command cd $(mktemp -d)
git clone https://github.com/digitalocean/csi-digitalocean
command cd csi-digitalocean/
git checkout b9ed6c2ea9ad85c5e4956bdfe418940bb5aa883a
rm go.sum
go mod tidy
command cd driver
time go list -export -deps -json -e . > /dev/null 2>&1
time go list -export -deps -json -e . > /dev/null 2>&1
time go list -export -deps -json -e . > /dev/null 2>&1
time go list -export -deps -json -e . > /dev/null 2>&1
) 2>&1 | tee output.txt

The "native" terminal run shows "average" speeds of ~300ms:

+ go list -export -deps -json -e .
real    0m1.524s
user    0m7.224s
sys 0m2.391s
+ go list -export -deps -json -e .
real    0m0.297s
user    0m0.420s
sys 0m0.898s
+ go list -export -deps -json -e .
real    0m0.300s
user    0m0.403s
sys 0m0.919s
+ go list -export -deps -json -e .
real    0m0.308s
user    0m0.402s
sys 0m0.968s

Whereas the Docker run shows "average" speeds of ~180ms:

+ go list -export -deps -json -e .

real    0m4.837s
user    0m12.070s
sys     0m5.030s
+ go list -export -deps -json -e .

real    0m0.186s
user    0m0.180s
sys     0m0.110s
+ go list -export -deps -json -e .

real    0m0.165s
user    0m0.160s
sys     0m0.120s
+ go list -export -deps -json -e .

real    0m0.187s
user    0m0.190s
sys     0m0.100s

What did you expect to see?

Similar run-times in each, possibly even faster times in the "native" OS X environment.

What did you see instead?

The "native" run-times are almost 70% longer.

cc @bcmills (and FYI @ianthehat for any go/packages side-effects)

@marwan-at-work

This comment has been minimized.

Contributor

marwan-at-work commented Nov 13, 2018

Here's what I got from running the above script on a 13' macOS (seirra 10.12.6)

Without Docker

+ godef -o 19358 -f controller.go
/var/folders/n4/35bgdnkd6vlgqrdzbyl0x1ycmhcndx/T/tmp.DVAXjerX/pkg/mod/github.com/container-storage-interface/spec@v0.3.0/lib/go/csi/v0/csi.pb.go:1233:1

real    0m10.234s
user    0m8.298s
sys    0m1.270s
+ godef -o 19358 -f controller.go
/var/folders/n4/35bgdnkd6vlgqrdzbyl0x1ycmhcndx/T/tmp.DVAXjerX/pkg/mod/github.com/container-storage-interface/spec@v0.3.0/lib/go/csi/v0/csi.pb.go:1233:1

real    0m1.276s
user    0m0.381s
sys    0m0.234s
+ godef -o 19358 -f controller.go
/var/folders/n4/35bgdnkd6vlgqrdzbyl0x1ycmhcndx/T/tmp.DVAXjerX/pkg/mod/github.com/container-storage-interface/spec@v0.3.0/lib/go/csi/v0/csi.pb.go:1233:1

real    0m1.295s
user    0m0.399s
sys    0m0.240s

With Docker

+ godef -o 19358 -f controller.go
/tmp/tmp.U5kJ73H8kO/pkg/mod/github.com/container-storage-interface/spec@v0.3.0/lib/go/csi/v0/csi.pb.go:1233:1

real    0m7.215s
user    0m10.540s
sys    0m2.270s
+ godef -o 19358 -f controller.go
/tmp/tmp.U5kJ73H8kO/pkg/mod/github.com/container-storage-interface/spec@v0.3.0/lib/go/csi/v0/csi.pb.go:1233:1

real    0m0.304s
user    0m0.250s
sys    0m0.110s
+ godef -o 19358 -f controller.go
/tmp/tmp.U5kJ73H8kO/pkg/mod/github.com/container-storage-interface/spec@v0.3.0/lib/go/csi/v0/csi.pb.go:1233:1

real    0m0.282s
user    0m0.230s
sys    0m0.110s

@myitcv myitcv changed the title from cmd/go: slow native performance with Mac OS X 10.14.1 to cmd/go: slow "native" performance with Mac OS X 10.14.1 and 10.12.6 Nov 13, 2018

@myitcv

This comment has been minimized.

Member

myitcv commented Nov 13, 2018

Sorry, forgot to cc @rsc in the original description

@agnivade

This comment has been minimized.

Member

agnivade commented Nov 13, 2018

Will a dtruss/strace log help here ? Just wondering if there is any OS level variable here.

@bcmills bcmills added this to the Go1.13 milestone Nov 13, 2018

@myitcv

This comment has been minimized.

Member

myitcv commented Nov 27, 2018

@jayconrod jayconrod self-assigned this Nov 27, 2018

@xmurobi

This comment has been minimized.

xmurobi commented Dec 4, 2018

The same problem on my windows 10:

What version of Go are you using (go version)?
go version go1.11 windows/amd64

What operating system and processor architecture are you using (go env)?
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\shuye\AppData\Local\go-build
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=G:\GO
set GOPROXY=
set GORACE=
set GOROOT=D:\Go
set GOTMPDIR=
set GOTOOLDIR=D:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=G:\GIT.HUB\marketstore\go.mod
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\shuye\AppData\Local\Temp\go-build930085358=/tmp/go-build -gno-record-gcc-switches

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment