Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/vgo: replace command appears to confuse vgo list #25427

Closed
jamie-digital opened this issue May 16, 2018 · 8 comments

Comments

@jamie-digital
Copy link

commented May 16, 2018

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.10 darwin/amd64 vgo:2018-02-20.1

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/path/to/Caches/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/path/to/gopath/go"
GORACE=""
GOROOT="/path/to/go"
GOTMPDIR=""
GOTOOLDIR="/path/to/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"

What did you do?

I used the replace command in my go.mod file to replace a particular module with a version I had forked. One of the changes I made in the forked version was to remove some unwanted dependencies. The module being replaced was the only source of these particular dependencies.

What did you expect to see?

I expected that vgo list -m would no longer list the dependencies mentioned above as the module replacement resulted in them no longer being dependencies of the build. Note that vgo list -m did pick up the replacement itself. Running vgo build -a -v did not list the dependencies, so I'm pretty sure they're not actually being used, but vgo list -m still shows them which is confusing.

@jamie-digital

This comment has been minimized.

Copy link
Author

commented May 17, 2018

I appreciate my original description was not very clear, so I'll give a more concrete example:

We have 3 initial modules; a, b, and c. They are:

$GOPATH/src/
           |→ a/
           |   |→ a.go
           |   |→ go.mod
           |
           |→ b/
           |   |→ b.go
           |   |→ go.mod
           |
           |→ c/
           |   |→ c.go
           |   |→ go.mod
  • c imports "fmt" and exports Println
  • b imports "c" and exports Println
  • a imports "b" as package main.

vgo list -m on a prints:

MODULE    VERSION
a         -
b         v1.0.0
c         v1.0.0

We then modify a/go.mod to add a line replace "b" v1.0.0 => "b2" v1.0.0 where b2 is a forked copy of b that does not depend on c.

If we then run vgo test all we get:

ok  	a (cached)
ok  	b (cached)

Which suggests that we're no longer depending on c, as we would expect. However, running go list -m shows:

MODULE    VERSION
a         -
b         v1.0.0
 => b2    v1.0.0
c         v1.0.0

I would not expect c to be listed any more as we don't depend upon it or build it.

@jamie-digital

This comment has been minimized.

Copy link
Author

commented May 17, 2018

More info that might be helpful that I just noticed is that if in the previous example we remove c from disk (both $GOPATH/src/v/cache/c and $GOPATH/src/v/c) and run vgo test all in a, we get:

vgo: finding c v1.0.0
ok  	a (cached)
ok  	b (cached)

Note that c is fetched but not downloaded.

@jamie-digital jamie-digital changed the title vgo: replace command appears to confuse vgo list x/vgo: replace command appears to confuse vgo list May 18, 2018

@gopherbot gopherbot added this to the vgo milestone May 18, 2018

@rsc

This comment has been minimized.

Copy link
Contributor

commented Jun 6, 2018

In this situation is C listed in your top-level go.mod file explicitly? List will include anything from the go.mod even if there is no import of it.

@jamie-digital

This comment has been minimized.

Copy link
Author

commented Jun 7, 2018

No, C isn't listed in go.mod. Another thing I noticed that might help in debugging is that vgo list -m shows the module C but vgo list all doesn't list the package c. (As a side note, thanks for making vgo list all work the way it does; it's phenomenally useful)

@jamie-digital

This comment has been minimized.

Copy link
Author

commented Jun 7, 2018

Edit: See next comment
I've just tried unsuccessfully to make a minimal reproduction of the issue. It's still occurring in a much larger repo but not in the tiny example I gave above (which I've recreated here: https://github.com/jamie-digital/a ). I'll try to reproduce it.

@jamie-digital

This comment has been minimized.

Copy link
Author

commented Jun 7, 2018

I'm afraid I've not been able to make a particularly minimal reproduction, but I've removed all of the private code and published a new repo https://github.com/jamie-digital/vgo-list-m-example . If you go into the example package you should get the following behaviour:

$ vgo list -m
MODULE                                               VERSION
github.com/jamie-digital/vgo-list-m-example/example  -
github.com/aws/aws-lambda-go                         v1.2.0
github.com/aws/aws-sdk-go                            v1.13.59
github.com/davecgh/go-spew                           v1.1.0
github.com/go-ini/ini                                v1.25.4
 => ../external/github.com/go-ini/ini
github.com/jmespath/go-jmespath                      v0.0.0-20160202185014-0b12d6b521d8
github.com/pmezard/go-difflib                        v1.0.0
github.com/stretchr/objx                             v0.0.0-20180106011353-facf9a85c22f
 => ../external/github.com/stretchr/objx
github.com/stretchr/testify                          v1.2.1
 => ../external/github.com/stretchr/testify
golang.org/x/net                                     v0.0.0-20180530234432-1e491301e022
golang.org/x/text                                    v0.3.0
gopkg.in/urfave/cli.v1                               v1.20.0-gopkgin-v1.20.0
$ vgo list all
github.com/aws/aws-lambda-go/lambda
github.com/aws/aws-lambda-go/lambda/messages
github.com/aws/aws-lambda-go/lambdacontext
github.com/aws/aws-sdk-go/aws
github.com/aws/aws-sdk-go/aws/awserr
github.com/aws/aws-sdk-go/aws/awsutil
github.com/aws/aws-sdk-go/aws/client
github.com/aws/aws-sdk-go/aws/client/metadata
github.com/aws/aws-sdk-go/aws/corehandlers
github.com/aws/aws-sdk-go/aws/credentials
github.com/aws/aws-sdk-go/aws/credentials/ec2rolecreds
github.com/aws/aws-sdk-go/aws/credentials/endpointcreds
github.com/aws/aws-sdk-go/aws/credentials/stscreds
github.com/aws/aws-sdk-go/aws/defaults
github.com/aws/aws-sdk-go/aws/ec2metadata
github.com/aws/aws-sdk-go/aws/endpoints
github.com/aws/aws-sdk-go/aws/request
github.com/aws/aws-sdk-go/aws/session
github.com/aws/aws-sdk-go/aws/signer/v4
github.com/aws/aws-sdk-go/awstesting
github.com/aws/aws-sdk-go/awstesting/mock
github.com/aws/aws-sdk-go/awstesting/unit
github.com/aws/aws-sdk-go/internal/sdkio
github.com/aws/aws-sdk-go/internal/sdkrand
github.com/aws/aws-sdk-go/internal/shareddefaults
github.com/aws/aws-sdk-go/private/protocol
github.com/aws/aws-sdk-go/private/protocol/ec2query
github.com/aws/aws-sdk-go/private/protocol/json/jsonutil
github.com/aws/aws-sdk-go/private/protocol/jsonrpc
github.com/aws/aws-sdk-go/private/protocol/query
github.com/aws/aws-sdk-go/private/protocol/query/queryutil
github.com/aws/aws-sdk-go/private/protocol/rest
github.com/aws/aws-sdk-go/private/protocol/restjson
github.com/aws/aws-sdk-go/private/protocol/restxml
github.com/aws/aws-sdk-go/private/protocol/xml/xmlutil
github.com/aws/aws-sdk-go/private/util
github.com/aws/aws-sdk-go/service/dynamodb
github.com/aws/aws-sdk-go/service/kinesis
github.com/aws/aws-sdk-go/service/route53
github.com/aws/aws-sdk-go/service/s3
github.com/aws/aws-sdk-go/service/sqs
github.com/aws/aws-sdk-go/service/sts
github.com/go-ini/ini
github.com/jamie-digital/vgo-list-m-example/example
github.com/jmespath/go-jmespath
github.com/stretchr/testify/assert
golang.org/x/net/http/httpguts
golang.org/x/net/http2
golang.org/x/net/http2/hpack
golang.org/x/net/idna
golang.org/x/text/internal/gen
golang.org/x/text/internal/testtext
golang.org/x/text/internal/ucd
golang.org/x/text/secure/bidirule
golang.org/x/text/transform
golang.org/x/text/unicode/bidi
golang.org/x/text/unicode/cldr
golang.org/x/text/unicode/norm
golang.org/x/text/unicode/rangetable

Note that github.com/pmezard/go-difflib is listed by vgo list -m but not by vgo list all and nor is it in any of the go.mod files (either the example package or any of the replacements).

I'm sorry the example is so complex. I've removed github.com/jamie-digital/a as it didn't reproduce the issue.

@agnivade agnivade removed the WaitingForInfo label Jun 7, 2018

@rsc

This comment has been minimized.

Copy link
Contributor

commented Jun 21, 2018

Thanks for the test case. I think vgo is working properly. I have some pending changes to expose the module graph, and if I try them on your test repo:

$ vgo mod -graph |grep pmezard
github.com/aws/aws-lambda-go@v1.2.0 github.com/pmezard/go-difflib@v1.0.0
$ 

That is, github.com/aws/aws-lambda-go is requiring github.com/pmezard/go-difflib, so it is correct to list it in the overall module list. It's also true that the module is not imported by anything in the aws-lambda-go repo, but the aws-lambda-go repo's Gopkg.lock mentions pmezard/go-difflib, and so we include it in the generated go.mod. Once aws-lambda-go converts to a native go.mod, they would presumably drop the explicit mention of pmezard/go-difflib, and then your test repo (if updated to use the new aws-lambda-go) would similarly drop it from the vgo list -m output.

@jamie-digital

This comment has been minimized.

Copy link
Author

commented Jun 21, 2018

That makes sense. I look forward to the graph tool!

@golang golang locked and limited conversation to collaborators Jun 21, 2019

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
4 participants
You can’t perform that action at this time.