Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/cmd/guru implements can take extreme amount of resources (e.g. when run from vscode) depending on working directory #41243

Open
Dieterbe opened this issue Sep 6, 2020 · 1 comment

Comments

@Dieterbe
Copy link
Contributor

@Dieterbe Dieterbe commented Sep 6, 2020

Hello, first of all, thanks for the work that's been put into guru. It's certainly been quite helpful to me over the years.
I know guru is no longer actively maintained (and I don't expect a resolution), but considering gopls is still in alpha, i thought it might be helpful to others (or to my future self) to document this issue.

What version of Go are you using (go version)?

go version go1.15.1 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/dieter/.cache/go-build"
GOENV="/home/dieter/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/dieter/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/dieter/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/dieter/go-build095063144=/tmp/go-build -gno-record-gcc-switches"

I found out that based on the working directory when guru implements is run, it can take an extreme amount of cpu and take a very long time.

In particular, consider these 3 types of runs:

1) cd /home/dieter && \
    guru -json implements '/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503'
2) cd /home/dieter/go/src/github.com/prometheus/prometheus && \
    guru -json implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
3) cd /home/dieter/go/src/github.com/prometheus/prometheus && \
    guru -json -scope github.com/prometheus/prometheus/... implements '/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503'

only 1) returns in a reasonable amount of time. 2) is how visual studio code invokes it (when CWD is $GOPATH/src/github.com/prometheus/prometheus), and 3) is my attempt to fix the problem with 2) by constraining the scope.

More info about each run:

Note:
I moved guru to guru-real and replaced guru with this script:

$ cat /home/dieter/go/bin/guru
#!/bin/bash
file="/home/dieter/guru-$(date '+%F_%T')"
echo "guru $@" > $file
env >> $file
guru-real "$@"

this allows me to track the exact environment of the process each time it is run (particularly when the parent is vscode and not my terminal), but it turns out the issue was reproducible perfectly from a terminal, so at this point it's no longer useful, but it does explain why i have mentions of "guru-real" in the pstree output below.

run 1

time /home/dieter/go/bin/guru -json implements '/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503'
{
	"type": {
		"name": "github.com/prometheus/prometheus/storage.Storage",
		"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:43:6",
		"kind": "interface"
	},
	"to": [
		{
			"name": "*github.com/prometheus/prometheus/cmd/prometheus.readyStorage",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/cmd/prometheus/main.go:898:6",
			"kind": "pointer"
		},
		{
			"name": "*github.com/prometheus/prometheus/storage.fanout",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/fanout.go:33:6",
			"kind": "pointer"
		},
		{
			"name": "*github.com/prometheus/prometheus/storage/remote.Storage",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/remote/storage.go:48:6",
			"kind": "pointer"
		},
		{
			"name": "*github.com/prometheus/prometheus/tsdb.DB",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/tsdb/db.go:120:6",
			"kind": "pointer"
		},
		{
			"name": "github.com/prometheus/prometheus/storage/fanout.errStorage",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/fanout/fanout_test.go:152:6",
			"kind": "struct"
		},
		{
			"name": "github.com/prometheus/prometheus/util/teststorage.testStorage",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/util/teststorage/storage.go:46:6",
			"kind": "struct"
		}
	],
	"from": [
		{
			"name": "github.com/cortexproject/cortex/vendor/github.com/prometheus/common/expfmt.Closer",
			"pos": "/home/dieter/go/src/github.com/cortexproject/cortex/vendor/github.com/prometheus/common/expfmt/encode.go:40:6",
			"kind": "interface"
		},
		{
			"name": "github.com/prometheus/prometheus/storage.Appendable",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:35:6",
			"kind": "interface"
		},
		{
			"name": "github.com/prometheus/prometheus/storage.Queryable",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:56:6",
			"kind": "interface"
		},
		{
			"name": "github.com/prometheus/prometheus/vendor/github.com/prometheus/common/expfmt.Closer",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/common/expfmt/encode.go:40:6",
			"kind": "interface"
		},
		{
			"name": "io.Closer",
			"pos": "/usr/lib/go/src/io/io.go:98:6",
			"kind": "interface"
		}
	]
}
/home/dieter/go/bin/guru -json implements   95.38s user 26.11s system 733% cpu 16.554 total
pstree -a -t -l | less
  |   |   `-guru /home/dieter/go/bin/guru -json implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
  |   |       `-guru-real -json implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
  |   |           `-13*[{guru-real}]

run 2

I didn't let it complete. I killed it after it ran for hours.

pstree -a -t -l | less
  |   |   `-guru /home/dieter/go/bin/guru -json implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
  |   |       `-guru-real -json implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/rog-go/cmd/share
  |   |           |   `-11*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/rog-go/cmd/hello
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/rog-go/cmd/shape
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/rog-go/cmd/peter-rabbit
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/rog-go/cmd/showdeps
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/rog-go/cmd/share2
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/go.tools/godoc/redirect
  |   |           |   `-11*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/go.tools/godoc/static
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/gogoprotobuf/plugin/unsafeunmarshaler
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/gogoprotobuf/plugin/marshalto
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/gogoprotobuf/plugin/populate
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/gogoprotobuf/plugin/stringer
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/gogoprotobuf/plugin/unmarshal
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/cluster/addons/kube-proxy
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/cluster/gce/manifests
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/cluster/juju/prereqs
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/cluster/kubemark/pre-existing
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/docs/yaml/kubectl
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/cmd/kube-apiserver/app
  |   |           |   `-11*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/cmd/kube-controller-manager/app
  |   |           |   `-11*[{go}]
  |   |           `-35*[{guru-real}]

run 3

time /home/dieter/go/bin/guru -json -scope github.com/prometheus/prometheus/... implements '/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503'
{
	"type": {
		"name": "github.com/prometheus/prometheus/storage.Storage",
		"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:43:6",
		"kind": "interface"
	},
	"to": [
		{
			"name": "*github.com/prometheus/prometheus/cmd/prometheus.readyStorage",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/cmd/prometheus/main.go:898:6",
			"kind": "pointer"
		},
		{
			"name": "*github.com/prometheus/prometheus/storage.fanout",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/fanout.go:33:6",
			"kind": "pointer"
		},
		{
			"name": "*github.com/prometheus/prometheus/storage/remote.Storage",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/remote/storage.go:48:6",
			"kind": "pointer"
		},
		{
			"name": "*github.com/prometheus/prometheus/tsdb.DB",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/tsdb/db.go:120:6",
			"kind": "pointer"
		},
		{
			"name": "github.com/prometheus/prometheus/storage/fanout.errStorage",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/fanout/fanout_test.go:152:6",
			"kind": "struct"
		},
		{
			"name": "github.com/prometheus/prometheus/util/teststorage.testStorage",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/util/teststorage/storage.go:46:6",
			"kind": "struct"
		}
	],
	"from": [
		{
			"name": "github.com/prometheus/common/expfmt.Closer",
			"pos": "/home/dieter/go/pkg/mod/github.com/prometheus/common@v0.10.0/expfmt/encode.go:40:6",
			"kind": "interface"
		},
		{
			"name": "github.com/prometheus/prometheus/storage.Appendable",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:35:6",
			"kind": "interface"
		},
		{
			"name": "github.com/prometheus/prometheus/storage.Queryable",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:56:6",
			"kind": "interface"
		},
		{
			"name": "github.com/prometheus/prometheus/vendor/github.com/prometheus/common/expfmt.Closer",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/common/expfmt/encode.go:40:6",
			"kind": "interface"
		},
		{
			"name": "io.Closer",
			"pos": "/usr/lib/go/src/io/io.go:98:6",
			"kind": "interface"
		}
	]
}
/home/dieter/go/bin/guru -json -scope github.com/prometheus/prometheus/...    5478.10s user 886.50s system 246% cpu 42:57.99 total
$ pstree -a -t -l
  |   |   `-guru /home/dieter/go/bin/guru -json -scope github.com/prometheus/prometheus/... implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
  |   |       `-guru-real -json -scope github.com/prometheus/prometheus/... implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- google.golang.org/protobuf/internal/descopts
  |   |           |   `-10*[{go}]
  |   |           |-(go)
  |   |           |   `-{go}
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- google.golang.org/protobuf/runtime/protoimpl
  |   |           |   `-11*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/apimachinery/pkg/util/intstr
  |   |           |   `-11*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/apimachinery/pkg/apis/meta/v1
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- google.golang.org/protobuf/internal/flags
  |   |           |   `-9*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/apimachinery/pkg/runtime/schema
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- github.com/prometheus/prometheus/vendor/k8s.io/client-go/plugin/pkg/client
  |   |           |   `-10*[{go}]
  |   |           `-19*[{guru-real}]
@gopherbot gopherbot added the Tools label Sep 6, 2020
@gopherbot gopherbot added this to the Unreleased milestone Sep 6, 2020
@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Sep 6, 2020

Thank you for this thorough investigation, but you are right to say that we will not be fixing guru issues. gopls is close to reaching beta, so it is the recommended alternative. If you are interested in using CLI tools, you can also try gopls implementation command, which will tell you both the implementations of a given interface or the interfaces implemented by a concrete type.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.