Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: document how to get 'go mod tidy' to keep an unused dependency #37352

Open
carnott-snap opened this issue Feb 21, 2020 · 6 comments
Open

cmd/go: document how to get 'go mod tidy' to keep an unused dependency #37352

carnott-snap opened this issue Feb 21, 2020 · 6 comments

Comments

@carnott-snap
Copy link

@carnott-snap carnott-snap commented Feb 21, 2020

What version of Go are you using (go version)?

$ go version
go version go1.13.8 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/user/.cache/go-build"
GOENV="/home/user/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/user/go"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/user/.local/share/umake/go/go-lang"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/user/.local/share/umake/go/go-lang/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/tmp/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build847180785=/tmp/go-build -gno-record-gcc-switches"

What did you do?

In light of the recent golang.org/x/crypto fix, I tried to enforce that my module use v0.0.0-20200220183623-bac4c82f6975:

// /tmp/go.mod
module tmp

go 1.13

require (
        golang.org/x/crypto v0.0.0-20200220183623-bac4c82f6975 // indirect
        lib v0.0.0-00010101000000-000000000000
)

replace lib => ./lib

golang.org/x/crypto is listed in my go.sum because a module I depend upon uses it:

// /tmp/go.sum
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550 h1:ObdrDkeb4kJdCP557AjRjq69pTHfNouLtWZG7j9rPN8=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
// /tmp/main.go
package main

import _ "lib"

func main() {}

It is assumed that I am not in control of the lib module:

// /tmp/lib/go.mod
module lib
  
go 1.13

require golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550
// /tmp/lib/lib.go
package lib
// /tmp/lib/sub/sub.go
package sub

import _ "golang.org/x/crypto/acme"

However it is not needed for build, as evidenced by go mod why. As such, when I go mod tidy, it gets dropped:

[user@localhost tmp]$ go mod why -m golang.org/x/crypto
# golang.org/x/crypto
(main module does not need module golang.org/x/crypto)
[user@localhost tmp]$ go mod tidy
[user@localhost tmp]$ cat go.mod

What did you expect to see?

// /tmp/go.mod
module test

go 1.13

require (
        golang.org/x/crypto v0.0.0-20200220183623-bac4c82f6975 // indirect
        lib v0.0.0-00010101000000-000000000000
)

replace lib => ./lib

What did you see instead?

// /tmp/go.mod
module test

go 1.13

require lib v0.0.0-00010101000000-000000000000

replace lib => ./lib
@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Feb 21, 2020

it is not needed for build, as evidenced by go mod why. As such, when I go mod tidy, it gets dropped

That is exactly what go mod tidy is documented to do (https://tip.golang.org/cmd/go/#hdr-Add_missing_and_remove_unused_modules):

it removes unused modules that don't provide any relevant packages.

Could you provide some more detail as to why you want to upgrade a module that does not provide any imported packages?

@bcmills bcmills added this to the Unplanned milestone Feb 21, 2020
@bcmills bcmills changed the title mod tidy drops upgraded, unused, indirect dependencies cmd/go: mod tidy drops upgraded, unused, indirect dependencies Feb 21, 2020
@carnott-snap

This comment has been minimized.

Copy link
Author

@carnott-snap carnott-snap commented Feb 21, 2020

That is exactly what go mod tidy is documented to do (https://tip.golang.org/cmd/go/#hdr-Add_missing_and_remove_unused_modules):

This is not just in tip; thanks for the pointer.

Could you provide some more detail as to why you want to upgrade a module that does not provide any imported packages?

  1. I have tooling that detects vulnerable modules in use from go.sum and go.mod. It flagged my go.sum, as it looks the same as if the module was actually being used for build.
  2. Since crypto is a transitive dependency, it would be easy for another developer to introduce a dependency upon sub without realising that they were bringing in the vulnerable version.
    • This is technically true for different modules too, but since we know lib depends on the vulnerable version, I would like a way to flag/upgrade it.
@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Feb 21, 2020

It flagged my go.sum, as it looks the same as if the module was actually being used for build.

Ah, #33008 could be related. For modules that do not contribute imported packages, the go.sum file is only supposed to contain a checksum for the go.mod file (not the full source code), but a bug in go mod tidy causes it to also retain a checksum for the source code if present.

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Feb 21, 2020

Since crypto is a transitive dependency, it would be easy for another developer to introduce a dependency upon sub without realising that they were bringing in the vulnerable version.

That's true, but we should probably provide some means to flag those regardless of how they are introduced. (That's #24031.)

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Feb 21, 2020

since we know lib depends on the vulnerable version, I would like a way to flag/upgrade it.

Note that if you really want to, you can introduce a dummy import of some package in order to convince go mod tidy to keep it. (See http://golang.org/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module.)

@bcmills bcmills changed the title cmd/go: mod tidy drops upgraded, unused, indirect dependencies cmd/go: document how to get 'go mod tidy' to keep an unused dependency Feb 21, 2020
@bcmills bcmills added NeedsFix and removed WaitingForInfo labels Feb 21, 2020
@bcmills bcmills modified the milestones: Unplanned, Backlog Feb 21, 2020
@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Feb 21, 2020

Since we have issues filed for the other two parts, let's focus this one on the third: you have a reason to want to retain an otherwise-unused requirement, and we have a mechanism for that (a dummy import), so we should probably make the documentation for that mechanism more discoverable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.