Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/gopls: no references are found due to `go list -deps` error #36770

Open
urandom opened this issue Jan 25, 2020 · 27 comments
Open

x/tools/gopls: no references are found due to `go list -deps` error #36770

urandom opened this issue Jan 25, 2020 · 27 comments
Labels
Milestone

Comments

@urandom
Copy link

@urandom urandom commented Jan 25, 2020

This error occurs when querying for references on a type with gopls@8fe064f8

[Trace - 00:19:52.525 AM] Sending request 'textDocument/references - (15)'.
Params: {"textDocument":{"uri":"file:///some/package/file.go"},"position":{"line":59,"character":26},"context":{"includeDeclaration":true}}


[Error - 00:19:52.525 AM] Received #15 go [-e -json -compiled=true -test=true -export=false -deps=true -find=false -- ./... builtin]: exit status 2: # pkg-config --cflags  -- vips vips vips vips
Package vips was not found in the pkg-config search path.
Perhaps you should add the directory containing `vips.pc'
to the PKG_CONFIG_PATH environment variable
Package 'vips', required by 'virtual:world', not found
Package 'vips', required by 'virtual:world', not found
Package 'vips', required by 'virtual:world', not found
Package 'vips', required by 'virtual:world', not found
pkg-config: exit status 1



[Trace - 00:20:03.050 AM] Sending request 'shutdown - (16)'.
Params: {}


[Trace - 00:20:03.051 AM] Received response 'shutdown - (16)' in 0ms.
Result: {}


[Trace - 00:20:03.051 AM] Sending notification 'exit'.
Params: null



As a result of this, no references are being displayed. Whereas the preferred solution would be to ignore the problematic dependency and search for references in the valid ones. It is also possible that other commands fail because of the same error as well.

@gopherbot gopherbot added this to the Unreleased milestone Jan 25, 2020
@stamblerre stamblerre modified the milestones: Unreleased, gopls/v0.3.0 Jan 25, 2020
@golang golang deleted a comment from gopherbot Jan 26, 2020
@stamblerre

This comment has been minimized.

Copy link
Contributor

@stamblerre stamblerre commented Jan 26, 2020

Can you run go list -e -json -compiled -deps -test ./path/to/directory/... for the directory at the root of your workspace? The reason I ask is because if go list failed then we may not have been able to get any metadata for any packages at all.

@urandom

This comment has been minimized.

Copy link
Author

@urandom urandom commented Jan 26, 2020

@stamblerre
There is a lot of metadata being returned, such as:

{
	"Dir": "/home/urandom/.go/pkg/mod/github.com/dgrijalva/jwt-go@v3.2.0+incompatible",
	"ImportPath": "github.com/dgrijalva/jwt-go",
	"Name": "jwt",
	"Doc": "Package jwt is a Go implementation of JSON Web Tokens: http://self-issued.info/docs/draft-jones-json-web-token.html See README.md for more info.",
	"Root": "/home/urandom/.go/pkg/mod/github.com/dgrijalva/jwt-go@v3.2.0+incompatible",
	"Module": {
		"Path": "github.com/dgrijalva/jwt-go",
		"Version": "v3.2.0+incompatible",
		"Time": "2018-03-08T23:13:08Z",
		"Dir": "/home/urandom/.go/pkg/mod/github.com/dgrijalva/jwt-go@v3.2.0+incompatible",
		"GoMod": "/home/urandom/.go/pkg/mod/cache/download/github.com/dgrijalva/jwt-go/@v/v3.2.0+incompatible.mod"
	},
	"DepOnly": true,
	"Stale": true,
	"StaleReason": "not installed but available in build cache",
	"GoFiles": [
		"claims.go",
		"doc.go",
		"ecdsa.go",
		"ecdsa_utils.go",
		"errors.go",
		"hmac.go",
		"map_claims.go",
		"none.go",
		"parser.go",
		"rsa.go",
		"rsa_pss.go",
		"rsa_utils.go",
		"signing_method.go",
		"token.go"
	],
	"CompiledGoFiles": [
		"claims.go",
		"doc.go",
		"ecdsa.go",
		"ecdsa_utils.go",
		"errors.go",
		"hmac.go",
		"map_claims.go",
		"none.go",
		"parser.go",
		"rsa.go",
		"rsa_pss.go",
		"rsa_utils.go",
		"signing_method.go",
		"token.go"
	],
	"Imports": [
		"bytes",
		"crypto",
		"crypto/ecdsa",
		"crypto/hmac",
		"crypto/rand",
		"crypto/rsa",
		"crypto/subtle",
		"crypto/x509",
		"encoding/base64",
		"encoding/json",
		"encoding/pem",
		"errors",
		"fmt",
		"math/big",
		"strings",
		"sync",
		"time"
	],
	"Deps": [
		"bufio",
		"bytes",
		"context",
		"crypto",
		"crypto/aes",
		"crypto/cipher",
		"crypto/des",
		"crypto/dsa",
		"crypto/ecdsa",
		"crypto/ed25519",
		"crypto/ed25519/internal/edwards25519",
		"crypto/elliptic",
		"crypto/hmac",
		"crypto/internal/randutil",
		"crypto/internal/subtle",
		"crypto/md5",
		"crypto/rand",
		"crypto/rsa",
		"crypto/sha1",
		"crypto/sha256",
		"crypto/sha512",
		"crypto/subtle",
		"crypto/x509",
		"crypto/x509/pkix",
		"encoding",
		"encoding/asn1",
		"encoding/base64",
		"encoding/binary",
		"encoding/hex",
		"encoding/json",
		"encoding/pem",
		"errors",
		"fmt",
		"hash",
		"internal/bytealg",
		"internal/cpu",
		"internal/fmtsort",
		"internal/nettrace",
		"internal/oserror",
		"internal/poll",
		"internal/race",
		"internal/reflectlite",
		"internal/singleflight",
		"internal/syscall/unix",
		"internal/testlog",
		"io",
		"io/ioutil",
		"math",
		"math/big",
		"math/bits",
		"math/rand",
		"net",
		"net/url",
		"os",
		"path/filepath",
		"reflect",
		"runtime",
		"runtime/cgo",
		"runtime/internal/atomic",
		"runtime/internal/math",
		"runtime/internal/sys",
		"sort",
		"strconv",
		"strings",
		"sync",
		"sync/atomic",
		"syscall",
		"time",
		"unicode",
		"unicode/utf16",
		"unicode/utf8",
		"unsafe",
		"vendor/golang.org/x/crypto/cryptobyte",
		"vendor/golang.org/x/crypto/cryptobyte/asn1",
		"vendor/golang.org/x/net/dns/dnsmessage"
	],
	"XTestGoFiles": [
		"ecdsa_test.go",
		"example_test.go",
		"hmac_example_test.go",
		"hmac_test.go",
		"http_example_test.go",
		"none_test.go",
		"parser_test.go",
		"rsa_pss_test.go",
		"rsa_test.go"
	],
	"XTestImports": [
		"bytes",
		"crypto/ecdsa",
		"crypto/rsa",
		"encoding/json",
		"fmt",
		"github.com/dgrijalva/jwt-go",
		"github.com/dgrijalva/jwt-go/request",
		"github.com/dgrijalva/jwt-go/test",
		"io",
		"io/ioutil",
		"log",
		"net",
		"net/http",
		"net/url",
		"reflect",
		"strings",
		"testing",
		"time"
	]
}

However, the command does exit with 2, and the following error (visible from the logs as well) is also present:

# pkg-config --cflags  -- vips vips vips vips
Package vips was not found in the pkg-config search path.
Perhaps you should add the directory containing `vips.pc'
to the PKG_CONFIG_PATH environment variable
Package 'vips', required by 'virtual:world', not found
Package 'vips', required by 'virtual:world', not found
Package 'vips', required by 'virtual:world', not found
Package 'vips', required by 'virtual:world', not found
pkg-config: exit status 1
@stamblerre

This comment has been minimized.

Copy link
Contributor

@stamblerre stamblerre commented Jan 26, 2020

Thanks for the logs! Once #36769 is resolved, can you share the output of gopls -rpc.trace -v check /path/to/file.go? I'll follow-up here again once that issue has been resolved.

@stamblerre

This comment has been minimized.

Copy link
Contributor

@stamblerre stamblerre commented Jan 27, 2020

#36769 is resolved, so if you could install gopls at master and share the output of gopls -rpc.trace -v check /path/to/file.go, that would be really helpful in diagnosing this issue. Thanks for your patience!

@urandom

This comment has been minimized.

Copy link
Author

@urandom urandom commented Jan 27, 2020

@stamblerre, there's a lot of output now, though I'm not sure I'll be able to share it since the repo is private. Instead, is there anything specific I should be looking for?

And it looks like some references are being returned now. Though unfortunately, only references within the definition package, but not others within other packages.

@stamblerre

This comment has been minimized.

Copy link
Contributor

@stamblerre stamblerre commented Jan 28, 2020

You should specifically look for a log line that contains text like this:

go/packages.Load
    snapshot = 0
    query = [./... builtin]
    packages = 3

I'd be interested to see how many packages are returned for that query. Any errors that are logged will also be helpful.

In general, when you first start up VS Code, if you see any log message that contains "diagnose: could not generate diagnostics for package", that would also be helpful.

@urandom

This comment has been minimized.

Copy link
Author

@urandom urandom commented Jan 28, 2020

Scratch my last comment. I was actually pulling 0.2.2. I just assumed that gopls version will always show the last stable version for some reason.

Now, with master, I'm still not getting any references. There are tons of errors related to go list again:

2020/01/28 09:09:58 Info:2020/01/28 09:09:58 Build info
----------
version master, built in $GOPATH mode

Go info
-------
go version go1.13.6 darwin/amd64

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/user/Library/Caches/go-build"
GOENV="/Users/user/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/user/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.13.6/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.13.6/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/user/projects/proj/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/ky/99mqlqrd2rd46chcz3tqmxkw0000gn/T/go-build330953888=/tmp/go-build -gno-record-gcc-switches -fno-common"
2020/01/28 09:10:00 Info:2020/01/28 09:10:00 go/packages.Load
	snapshot = 0
	query = [./... builtin]
	packages = 0
2020/01/28 09:10:00 diagnose: no workspace packages: go [-e -json -compiled=true -test=true -export=false -deps=true -find=false -- ./... builtin]: exit status 1: go build github.com/foomo/bimg: invalid flag in pkg-config --cflags: -Xpreprocessor

	directory = 0x1711a60
2020/01/28 09:10:00 Error:2020/01/28 09:10:00 diagnose: no workspace packages: go [-e -json -compiled=true -test=true -export=false -deps=true -find=false -- ./... builtin]: exit status 1: go build github.com/foomo/bimg: invalid flag in pkg-config --cflags: -Xpreprocessor

	directory = 0x1711a60
2020/01/28 09:10:00 diagnose: no workspace packages: go [-e -json -compiled=true -test=true -export=false -deps=true -find=false -- ./... builtin]: exit status 1: go build github.com/foomo/bimg: invalid flag in pkg-config --cflags: -Xpreprocessor

	directory = 0x1711a60
2020/01/28 09:10:00 Error:2020/01/28 09:10:00 diagnose: no workspace packages: go [-e -json -compiled=true -test=true -export=false -deps=true -find=false -- ./... builtin]: exit status 1: go build github.com/foomo/bimg: invalid flag in pkg-config --cflags: -Xpreprocessor

	directory = 0x1711a60
@stamblerre

This comment has been minimized.

Copy link
Contributor

@stamblerre stamblerre commented Jan 28, 2020

Thanks for sharing this log!

/cc @matloob for go/packages + cgo insight

@matloob

This comment has been minimized.

Copy link
Contributor

@matloob matloob commented Jan 28, 2020

In general, go list needs to be able to work for our tools to work. Does your project build with go build? Do you use a Makefile (or other build system) that sets the cgo environment variables? If so, you'd need to set the environment variables for gopls.

@urandom

This comment has been minimized.

Copy link
Author

@urandom urandom commented Jan 29, 2020

@matloob

In general, go list needs to be able to work for our tools to work.

go list works fine, though it would be an improvement to report the error as json, part of the object that produced it. Beside exiting with 2 and printing a cgo-related error for a dependency that is used in some dark corner of the project, it also produces more than 30000 lines of json code. I believe the problem lies in gopls, which ignores all of that json due to the error.

Does your project build with go build?

go build works fine as well. I'm sure there's a main package somewhere that ends up using the problematic dependency and probably won't build without help, but that's not a problem for the vast majority of the code and the plethora of main packages in the module.

Do you use a Makefile (or other build system) that sets the cgo environment variables?

No

@stamblerre

This comment has been minimized.

Copy link
Contributor

@stamblerre stamblerre commented Jan 29, 2020

@matloob: Will it be possible for go/packages to ignore this form of error? I imagine this would be an extension of the workaround here?

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Jan 30, 2020

Change https://golang.org/cl/217078 mentions this issue: go/packages: work around pkg-config errors in go list

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Jan 30, 2020

Change https://golang.org/cl/217083 mentions this issue: go/packages: work around pkg-config errors in go list

@stamblerre

This comment has been minimized.

Copy link
Contributor

@stamblerre stamblerre commented Jan 30, 2020

@urandom: Can you confirm that this issue is fixed by downloading gopls at master? The command to do so is GO111MODULE=on go get golang.org/x/tools/gopls@master golang.org/x/tools@master.

@urandom

This comment has been minimized.

Copy link
Author

@urandom urandom commented Jan 30, 2020

@stamblerre after updating gopls to master, i can now see some references. though it appears that they are only from the current package.

The output of the check subcommand now looks like this:

2020/01/30 22:31:24 Info:2020/01/30 22:31:24 Build info
----------
golang.org/x/tools/gopls master
    golang.org/x/tools/gopls@v0.1.8-0.20200130203232-449c356b79e5 h1:QFfk5i7qHxUADE2pe8snS1epWbFguJGJ7KbdtVDiANw=
    github.com/BurntSushi/toml@v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=
    github.com/sergi/go-diff@v1.0.0 h1:Kpca3qRNrduNnOQeazBd0ysaKrUJiIuISHxogkT9RPQ=
    golang.org/x/mod@v0.1.1-0.20191105210325-c90efee705ee h1:WG0RUwxtNT4qqaXX3DPA8zHFNm/D9xaBpxzHt1WcA/E=
    golang.org/x/sync@v0.0.0-20190423024810-112230192c58 h1:8gQV6CLnAEikrhgkHFbMAEhagSSnXWGV915qUMm9mrU=
    golang.org/x/tools@v0.0.0-20200130193611-71629799394e h1:XuSNSStN96LDcnwKA/D7kyYMR7y8HVIh/0g5rYU4l9o=
    golang.org/x/xerrors@v0.0.0-20191011141410-1b5146add898 h1:/atklqdjdhuosWIl6AIbOeHJjicWYPqR9bpxqxYG2pA=
    honnef.co/go/tools@v0.0.1-2019.2.3 h1:3JgtbtFHMiCmsznwGVTUWbgGov+pVqnlf1dEJTNAXeM=
    mvdan.cc/xurls/v2@v2.1.0 h1:KaMb5GLhlcSX+e+qhbRJODnUUBvlw01jt4yrjFIHAuA=

Go info
-------
go version go1.13.7 linux/amd64

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/urandom/.cache/go-build"
GOENV="/home/urandom/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/urandom/.go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/urandom/Projects/proj/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build274623648=/tmp/go-build -gno-record-gcc-switches"
2020/01/30 22:31:26 initial workspace load failed: go [-e -json -compiled=true -test=true -export=false -deps=true -find=false -- ./... builtin]: exit status 2: # pkg-config --cflags  -- vips vips vips vips
Package vips was not found in the pkg-config search path.
Perhaps you should add the directory containing `vips.pc'
to the PKG_CONFIG_PATH environment variable
Package 'vips', required by 'virtual:world', not found
Package 'vips', required by 'virtual:world', not found
Package 'vips', required by 'virtual:world', not found
Package 'vips', required by 'virtual:world', not found
pkg-config: exit status 1

2020/01/30 22:31:26 Info:2020/01/30 22:31:26 go/packages.Load
	snapshot = 0
	query = [./... builtin]
	packages = 0
2020/01/30 22:31:26 Error:2020/01/30 22:31:26 initial workspace load failed: go [-e -json -compiled=true -test=true -export=false -deps=true -find=false -- ./... builtin]: exit status 2: # pkg-config --cflags  -- vips vips vips vips
Package vips was not found in the pkg-config search path.
Perhaps you should add the directory containing `vips.pc'
to the PKG_CONFIG_PATH environment variable
Package 'vips', required by 'virtual:world', not found
Package 'vips', required by 'virtual:world', not found
Package 'vips', required by 'virtual:world', not found
Package 'vips', required by 'virtual:world', not found
pkg-config: exit status 1
2020/01/30 22:31:26 Info:2020/01/30 22:31:26 go/packages.Load
	snapshot = 1
	query = [file=/home/urandom/Projects/proj/file1.go]
	packages = 2
2020/01/30 22:31:26 Info:2020/01/30 22:31:26 go/packages.Load
	snapshot = 1
	package = pkg/
	files = [/home/urandom/Projects/proj/file2.go /home/urandom/Projects/proj/file1.go]
2020/01/30 22:31:26 Info:2020/01/30 22:31:26 go/packages.Load
	snapshot = 1
	package = pkg/
	files = [/home/urandom/Projects/proj/file2.go /home/urandom/Projects/proj/file1.go /home/urandom/Projects/proj/file2_test.go /home/urandom/Projects/proj/file1_test.go]

@stamblerre

This comment has been minimized.

Copy link
Contributor

@stamblerre stamblerre commented Jan 30, 2020

Ah, looks like Michael's CL still didn't fix your error. @matloob: Can you take a look?

@stamblerre stamblerre reopened this Jan 30, 2020
@matloob

This comment has been minimized.

Copy link
Contributor

@matloob matloob commented Jan 30, 2020

I'm a bit stumped as to why this is happening. It would be really helpful if there was a repo we could try to reproduce this with locally so we can try to debug this. Is that something you could give us?

@urandom

This comment has been minimized.

Copy link
Author

@urandom urandom commented Jan 31, 2020

Some more information:

On a mac, the error message doesn't begin with a '#', so the workaround will not work there. This is what go list's stderr contains there:

go build github.com/foomo/bimg: invalid flag in pkg-config --cflags: -Xpreprocessor

Also, it appears that this github.com/foomo/bimg is responsible for the whole pkg-config mess. The following trivial example reproduces the error:

package main

import (
	"fmt"

	_ "github.com/foomo/bimg"
)

func main() {
	fmt.Println("test")
}
@matloob

This comment has been minimized.

Copy link
Contributor

@matloob matloob commented Jan 31, 2020

Thanks for the repro, I'll see what's going on.

The weird thing is that I'm getting the "#" in my stderr:

$ go list -json -compiled uandom.go >/dev/null
# pkg-config --cflags  -- vips vips vips vips
Package vips was not found in the pkg-config search path.
Perhaps you should add the directory containing `vips.pc'
to the PKG_CONFIG_PATH environment variable
No package 'vips' found
Package vips was not found in the pkg-config search path.
Perhaps you should add the directory containing `vips.pc'
to the PKG_CONFIG_PATH environment variable
No package 'vips' found
Package vips was not found in the pkg-config search path.
Perhaps you should add the directory containing `vips.pc'
to the PKG_CONFIG_PATH environment variable
No package 'vips' found
Package vips was not found in the pkg-config search path.
Perhaps you should add the directory containing `vips.pc'
to the PKG_CONFIG_PATH environment variable
No package 'vips' found
pkg-config: exit status 1
@matloob

This comment has been minimized.

Copy link
Contributor

@matloob matloob commented Jan 31, 2020

Okay, I can't reproduce this locally. I'm wondering if the version of gopls@master you built also pinned tools at master? (It's not enough to get gopls@master because it pins tools to an old pseudoversion in its go.mod). What happens if you upgrade to gopls@latest (which includes the newest version of tools in its go.mod), and try the command then?

@urandom

This comment has been minimized.

Copy link
Author

@urandom urandom commented Feb 1, 2020

This is now on linux with gopls@latest when i invoke the references subcommand:

2020/02/01 16:17:19 Info:2020/02/01 16:17:19 Build info
----------
golang.org/x/tools/gopls v0.3.0
    golang.org/x/tools/gopls@v0.3.0 h1:l9KKK1/n6CIbfgaUvHBWAvCfOxcl1N+KSOK79OlPIao=
    github.com/BurntSushi/toml@v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=
    github.com/sergi/go-diff@v1.0.0 h1:Kpca3qRNrduNnOQeazBd0ysaKrUJiIuISHxogkT9RPQ=
    golang.org/x/mod@v0.1.1-0.20191105210325-c90efee705ee h1:WG0RUwxtNT4qqaXX3DPA8zHFNm/D9xaBpxzHt1WcA/E=
    golang.org/x/sync@v0.0.0-20190423024810-112230192c58 h1:8gQV6CLnAEikrhgkHFbMAEhagSSnXWGV915qUMm9mrU=
    golang.org/x/tools@v0.0.0-20200130224948-02f1738cbe39 h1:5ERHXLQfA0b8cHOwaOfWaaGekrA4+Ka/N74zilLnsIk=
    golang.org/x/xerrors@v0.0.0-20191011141410-1b5146add898 h1:/atklqdjdhuosWIl6AIbOeHJjicWYPqR9bpxqxYG2pA=
    honnef.co/go/tools@v0.0.1-2019.2.3 h1:3JgtbtFHMiCmsznwGVTUWbgGov+pVqnlf1dEJTNAXeM=
    mvdan.cc/xurls/v2@v2.1.0 h1:KaMb5GLhlcSX+e+qhbRJODnUUBvlw01jt4yrjFIHAuA=

Go info
-------
go version go1.13.7 linux/amd64

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/urandom/.cache/go-build"
GOENV="/home/urandom/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/urandom/.go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/urandom/Projects/proj/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build866492451=/tmp/go-build -gno-record-gcc-switches"
2020/02/01 16:17:21 initial workspace load failed: go [-e -json -compiled=true -test=true -export=false -deps=true -find=false -- ./... builtin]: exit status 2: # pkg-config --cflags  -- vips vips vips vips
Package vips was not found in the pkg-config search path.
Perhaps you should add the directory containing `vips.pc'
to the PKG_CONFIG_PATH environment variable
Package 'vips', required by 'virtual:world', not found
Package 'vips', required by 'virtual:world', not found
Package 'vips', required by 'virtual:world', not found
Package 'vips', required by 'virtual:world', not found
pkg-config: exit status 1

2020/02/01 16:17:21 Error:2020/02/01 16:17:21 initial workspace load failed: go [-e -json -compiled=true -test=true -export=false -deps=true -find=false -- ./... builtin]: exit status 2: # pkg-config --cflags  -- vips vips vips vips
Package vips was not found in the pkg-config search path.
Perhaps you should add the directory containing `vips.pc'
to the PKG_CONFIG_PATH environment variable
Package 'vips', required by 'virtual:world', not found
Package 'vips', required by 'virtual:world', not found
Package 'vips', required by 'virtual:world', not found
Package 'vips', required by 'virtual:world', not found
pkg-config: exit status 1
2020/02/01 16:17:21 Info:2020/02/01 16:17:21 go/packages.Load
	snapshot = 0
	query = [./... builtin]
	packages = 0
2020/02/01 16:17:22 Info:2020/02/01 16:17:22 go/packages.Load
	snapshot = 1
	query = [file=/home/urandom/Projects/proj/file1.go]
	packages = 2
2020/02/01 16:17:22 Info:2020/02/01 16:17:22 go/packages.Load
	snapshot = 1
	package = pkg/
	files = [/home/urandom/Projects/proj/file2.go /home/urandom/Projects/proj/file1.go]
2020/02/01 16:17:22 Info:2020/02/01 16:17:22 go/packages.Load
	snapshot = 1
	package = pkg/
	files = [/home/urandom/Projects/proj/file2.go /home/urandom/Projects/proj/file1.go /home/urandom/Projects/proj/file2_test.go /home/urandom/Projects/proj/file1_test.go]
/home/urandom/Projects/proj/file2.go:15:13-22
/home/urandom/Projects/proj/file1.go:113:10-19
/home/urandom/Projects/proj/file1.go:19:58-67
/home/urandom/Projects/proj/file1.go:22:10-19
/home/urandom/Projects/proj/file1.go:27:10-19
/home/urandom/Projects/proj/file1.go:30:9-18
/home/urandom/Projects/proj/file1.go:36:9-18
/home/urandom/Projects/proj/file1.go:45:9-18
/home/urandom/Projects/proj/file1.go:51:9-18
/home/urandom/Projects/proj/file1.go:55:9-18
/home/urandom/Projects/proj/file1.go:59:9-18
/home/urandom/Projects/proj/file1.go:65:9-18
/home/urandom/Projects/proj/file1.go:71:9-18
/home/urandom/Projects/proj/file1.go:81:9-18
/home/urandom/Projects/proj/file1.go:97:9-18

The object ref I'm trying to find in file1 should be referenced in a lot of packages, but only the local references are found.

@stamblerre stamblerre modified the milestones: gopls/v1.0.0, gopls/v0.4.0 Feb 2, 2020
@stamblerre

This comment has been minimized.

Copy link
Contributor

@stamblerre stamblerre commented Feb 2, 2020

Yeah, the cause seems to be the same here - we're failing to load your entire workspace because of the cgo dependency, which is why the references from other packages don't appear. But @matloob, it looks like the stderr printed does contain a #, so I wonder why the go/packages workaround isn't kicking in here?

@urandom

This comment has been minimized.

Copy link
Author

@urandom urandom commented Feb 2, 2020

Yeah, the cause seems to be the same here - we're failing to load your entire workspace because of the cgo dependency, which is why the references from other packages don't appear. But @matloob, it looks like the stderr printed does contain a #, so I wonder why the go/packages workaround isn't kicking in here?

Keep in mind that this is on Linux. On a Mac, stderr does not start with a # as it appears to be a different error altogether.

BTW, why the need for workarounds? If stdout contains valid data, why can't it just be used, and the stderr logged instead?

@stamblerre

This comment has been minimized.

Copy link
Contributor

@stamblerre stamblerre commented Feb 4, 2020

I'm going to continue investigating this. Just jotting down some quick notes as I notice things - I am only able to reproduce with a clean modcache. Once the "github.com/foomo/bimg" dependency is in my modcache, things work as expected, but the initial workspace load fails with an empty modcache.

@urandom: Regarding the workarounds, you can see here that a lot of work has to be done to determine if the go list failure is catastrophic or not. Otherwise, we may end up using invalid metadata for the package.

@stamblerre

This comment has been minimized.

Copy link
Contributor

@stamblerre stamblerre commented Feb 4, 2020

Looks like the work-around fails for a clean modcache because stderr looks like this:

go: downloading github.com/foomo/bimg v1.0.18
# pkg-config --cflags  -- vips vips vips vips
Package vips was not found in the pkg-config search path.
Perhaps you should add the directory containing `vips.pc'
to the PKG_CONFIG_PATH environment variable
No package 'vips' found
Package vips was not found in the pkg-config search path.
Perhaps you should add the directory containing `vips.pc'
to the PKG_CONFIG_PATH environment variable
No package 'vips' found
Package vips was not found in the pkg-config search path.
Perhaps you should add the directory containing `vips.pc'
to the PKG_CONFIG_PATH environment variable
No package 'vips' found
Package vips was not found in the pkg-config search path.
Perhaps you should add the directory containing `vips.pc'
to the PKG_CONFIG_PATH environment variable
No package 'vips' found
pkg-config: exit status 1

I don't think that go/packages should work-around that, and instead, we should simply retry the initial workspace load (#36854).

@urandom: Can you confirm that this issue only reproduces when github.com/foomo/bimg does not exist in your modcache, i.e. when $GOPATH/pkg/mod/github.com/foomo/bimg@v1.0.18 does not yet exist?

@urandom

This comment has been minimized.

Copy link
Author

@urandom urandom commented Feb 5, 2020

@stamblerre

It looks like go list does not produce any errors once the dep is in the modcache, regardless of whether it can be built or not.

@stamblerre

This comment has been minimized.

Copy link
Contributor

@stamblerre stamblerre commented Feb 5, 2020

Perfect, thanks for letting me know. I'll leave this issue open until #36854 is resolved and then confirm that it fixes things.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.