New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: building requires otherwise-unused .mod files to be present #29410

Closed
stapelberg opened this Issue Dec 24, 2018 · 9 comments

Comments

Projects
None yet
4 participants
@stapelberg
Copy link
Contributor

stapelberg commented Dec 24, 2018

What version of Go are you using (go version)?

$ go version
go version go1.12beta1 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/michael/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/tmp/gp-repro"
GOPROXY=""
GORACE=""
GOROOT="/home/michael/sdk/go1.12beta1"
GOTMPDIR=""
GOTOOLDIR="/home/michael/sdk/go1.12beta1/pkg/tool/linux_amd64"
GCCGO="/usr/bin/gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/tmp/repro/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build500484946=/tmp/go-build -gno-record-gcc-switches"

What did you do?

% export GOPATH=/tmp/gp-repro
% mkdir /tmp/repro
% cat > repro.go <<'EOT'
package main

import (
	"fmt"

	"golang.org/x/exp/mmap"
	"gonum.org/v1/gonum/unit"
)

func main() {
	fmt.Printf("%v, %v", mmap.ReaderAt{}, unit.Meter)
}
EOT
% go mod init repro
% go build
go: finding gonum.org/v1/gonum v0.0.0-20181223131727-b545e3e77e9e
go: finding golang.org/x/exp v0.0.0-20181221233300-b68661188fbf
go: finding golang.org/x/exp v0.0.0-20180321215751-8460e604b9de
go: finding golang.org/x/tools v0.0.0-20180525024113-a5b4c53f6e8b
go: downloading gonum.org/v1/gonum v0.0.0-20181223131727-b545e3e77e9e
go: downloading golang.org/x/exp v0.0.0-20181221233300-b68661188fbf
go: extracting golang.org/x/exp v0.0.0-20181221233300-b68661188fbf
go: extracting gonum.org/v1/gonum v0.0.0-20181223131727-b545e3e77e9e

As you can see, two versions of golang.org/x/exp are involved here: the repro module uses the latest (v0.0.0-20181221233300-b68661188fbf), whereas the gonum.org/v1/gonum module specifies an older version (v0.0.0-20180321215751-8460e604b9de).

In the build itself, the latest version is picked:

% go list -m -json all
{
	"Path": "repro",
	"Main": true,
	"Dir": "/tmp/repro",
	"GoMod": "/tmp/repro/go.mod",
	"GoVersion": "1.12"
}
{
	"Path": "golang.org/x/exp",
	"Version": "v0.0.0-20181221233300-b68661188fbf",
	"Time": "2018-12-21T23:33:00Z",
	"Dir": "/tmp/gp-repro/pkg/mod/golang.org/x/exp@v0.0.0-20181221233300-b68661188fbf",
	"GoMod": "/tmp/gp-repro/pkg/mod/cache/download/golang.org/x/exp/@v/v0.0.0-20181221233300-b68661188fbf.mod"
}
{
	"Path": "golang.org/x/tools",
	"Version": "v0.0.0-20180525024113-a5b4c53f6e8b",
	"Time": "2018-05-25T02:41:13Z",
	"Indirect": true,
	"GoMod": "/tmp/gp-repro/pkg/mod/cache/download/golang.org/x/tools/@v/v0.0.0-20180525024113-a5b4c53f6e8b.mod"
}
{
	"Path": "gonum.org/v1/gonum",
	"Version": "v0.0.0-20181223131727-b545e3e77e9e",
	"Time": "2018-12-23T13:17:27Z",
	"Dir": "/tmp/gp-repro/pkg/mod/gonum.org/v1/gonum@v0.0.0-20181223131727-b545e3e77e9e",
	"GoMod": "/tmp/gp-repro/pkg/mod/cache/download/gonum.org/v1/gonum/@v/v0.0.0-20181223131727-b545e3e77e9e.mod"
}

However, when preventing internet access by pointing GOPROXY to an unused port, I can see that the old version of golang.org/x/exp needs to be present for the build to succeed:

% rm repro && GOPROXY=http://localhost:1111/ go build
# works
% rm /tmp/gp-repro/pkg/mod/cache/download/golang.org/x/exp/@v/v0.0.0-20180321215751-8460e604b9de.*
% rm repro && GOPROXY=http://localhost:1111/ go build
go: finding golang.org/x/exp v0.0.0-20180321215751-8460e604b9de
go: golang.org/x/exp@v0.0.0-20180321215751-8460e604b9de: Get http://localhost:1111/golang.org/x/exp/@v/v0.0.0-20180321215751-8460e604b9de.info: dial tcp [::1]:1111: connect: connection refused
go: error loading module requirements
% echo 'module golang.org/x/exp' > /tmp/gp-repro/pkg/mod/cache/download/golang.org/x/exp/@v/v0.0.0-20180321215751-8460e604b9de.mod
% GOPROXY=http://localhost:1111/ go build                                                                                         
# works

This came as a surprise to me, so I wanted to check in and see if this was intentional or an oversight. Given that the .mod file only contains “module golang.org/x/exp”, it seems to me that the file could easily be synthesized?

I discovered this issue because I was experimenting with packaging Go modules in Linux distributions (e.g. Debian). It would be a significant downside if we needed to provide (and therefore maintain!) all versions that a build ever references, instead of just the versions which are selected in the end.

Thanks,

@mvdan

This comment has been minimized.

Copy link
Member

mvdan commented Dec 24, 2018

It might take a while for this to get a proper reply, as most engineers will be on holiday this week :)

/cc @bcmills @myitcv @thepudds

@thepudds

This comment has been minimized.

Copy link

thepudds commented Dec 24, 2018

@stapelberg two quick comments:

  1. you might want to set GOPROXY=off (rather than using a non-existent port). That means go commands in module mode are disallowed from using the network for dependencies. I don't know if that explains your symptoms (and probably it does not), but you might get more standard behavior in general out of GOPROXY=off.

2. you could try with the go 1.12 beta. I think there was at least one fix in 1.12 for avoiding hitting the network unnecessarily (though it might have had to do with replace, which might not help here).

edit: sorry, I see you are using the 1.12 beta.

@thepudds

This comment has been minimized.

Copy link

thepudds commented Dec 24, 2018

Setting GOPROXY=off and attempting to reproduce gives a slightly different error, but still fails:

$ go1.12beta1 build
go: golang.org/x/exp@v0.0.0-20180321215751-8460e604b9de: module lookup disabled by GOPROXY=off
go: error loading module requirements

Someone else might need to comment if this is a bug, vs. necessary behavior, vs. perhaps a not-yet-implemented feature could avoid this.

@stapelberg you mentioned:

It would be a significant downside if we needed to provide (and therefore maintain!) all versions that a build ever references, instead of just the versions which are selected in the end.

I am just looking at this quickly, but is it just meta data about the old version 20180321215751-8460e604b9de that currently seems to be required? E.g., after a clean build (with GOPROXY unset):

$ ls $GOPATH/pkg/mod/cache/download/golang.org/x/exp/@v                      
list
v0.0.0-20180321215751-8460e604b9de.info
v0.0.0-20180321215751-8460e604b9de.mod
v0.0.0-20181221233300-b68661188fbf.info
v0.0.0-20181221233300-b68661188fbf.mod
v0.0.0-20181221233300-b68661188fbf.zip
v0.0.0-20181221233300-b68661188fbf.ziphash

and:

$ ls $GOPATH/pkg/mod/golang.org/x
exp@v0.0.0-20181221233300-b68661188fbf

or:

$ cd $GOPATH/pkg/mod
$ find -name "*8460e604b9de*"
./cache/download/golang.org/x/exp/@v/v0.0.0-20180321215751-8460e604b9de.info
./cache/download/golang.org/x/exp/@v/v0.0.0-20180321215751-8460e604b9de.mod

@stapelberg

This comment has been minimized.

Copy link
Contributor

stapelberg commented Dec 24, 2018

I am just looking at this quickly, but is it just meta data about the old version 20180321215751-8460e604b9de that currently seems to be required? E.g., after a clean build (with GOPROXY unset):

Yes, it’s just the .mod file. Hence my suspicion that this is unintended.

@bcmills

This comment has been minimized.

Copy link
Member

bcmills commented Jan 17, 2019

This is working as designed. Every go.mod file in the transitive requirement graph — even for versions that we know will not be selected — impacts the constraints on which versions we can ultimately choose, so we have to load those files in order to compute the final versions.

The good news is, go.mod files are small and cacheable, so the impact on build times should usually be negligible.

See the cross test-cases for the MVS package for examples:

# Cross-dependency between D and E.
# No matter how it arises, should get result of merging all build lists via max,
# which leads to including both D2 and E2.
name: cross1
A: B C
B: D1
C: D2
D1: E2
D2: E1
build A: A B C D2 E2
name: cross1V
A: B2 C D2 E1
B1:
B2: D1
C: D2
D1: E2
D2: E1
build A: A B2 C D2 E2
name: cross1U
A: B1 C
B1:
B2: D1
C: D2
D1: E2
D2: E1
build A: A B1 C D2 E1
upgrade A B2: A B2 C D2 E2
name: cross1R
A: B C
B: D2
C: D1
D1: E2
D2: E1
build A: A B C D2 E2
name: cross1X
A: B C
B: D1 E2
C: D2
D1: E2
D2: E1
build A: A B C D2 E2
name: cross2
A: B D2
B: D1
D1: E2
D2: E1
build A: A B D2 E2
name: cross2X
A: B D2
B: D1 E2
C: D2
D1: E2
D2: E1
build A: A B D2 E2
name: cross3
A: B D2 E1
B: D1
D1: E2
D2: E1
build A: A B D2 E2
name: cross3X
A: B D2 E1
B: D1 E2
D1: E2
D2: E1
build A: A B D2 E2

@bcmills bcmills closed this Jan 17, 2019

@stapelberg

This comment has been minimized.

Copy link
Contributor

stapelberg commented Jan 17, 2019

This is working as designed. Every go.mod file in the transitive requirement graph — even for versions that we know will not be selected — impacts the constraints on which versions we can ultimately choose, so we have to load those files in order to compute the final versions.

Can you elaborate on the ways in which these files can impact the version selection? Specifically, are you saying it is not sufficient to synthesize these files?

The good news is, go.mod files are small and cacheable, so the impact on build times should usually be negligible.

That’s not my concern, though. In Linux distributions such as Debian, we strive to have precisely one version of a specific module packaged. We can’t keep old packages around just for their .mod files.

If this is really working as intended with no wiggle room, we’ll need to work around that and either modify .mod files at package-time (seems more labor-intensive, but haven’t thought it through entirely) or synthesizing .mod files at build-time (ugh).

@bcmills

This comment has been minimized.

Copy link
Member

bcmills commented Jan 17, 2019

In Linux distributions such as Debian, we strive to have precisely one version of a specific module packaged. We can’t keep old packages around just for their .mod files.

Sorry, but Go's version selection algorithm intentionally prioritizes reproducibility over recency: when you build a module-enabled binary, you use the specific versions of the modules it was tested against, not whatever newer version happens to be installed on your system.

Now that #27859 is fixed, you can probably make something work using wildcard replace directives, but the resulting builds will not be as reproducible as ordinary module-enabled builds.

@thepudds

This comment has been minimized.

Copy link

thepudds commented Jan 17, 2019

@stapelberg Is this in the context of what I think in the past might have been described as "more or less one giant GOPATH" for Debian packaging of Go libraries, or is this more in the context of trying to use Go modules to break out of that (or perhaps nothing to do with any of that)?

I am not sure how much flexibility you have at package time vs. build time, how that changes for a package vs. a binary, the level of control around setting env variables, or your ability to use an auxiliary directory that contains auxiliary data like go.mod files...

...but depending on your constraints, you might be able to do some variety of using GOPROXY=file:/// and/or temporarily setting GOPATH (which controls the location of GOPATH/pkg/mod cache location). That might not help, but here is a concrete example (from @myitcv):

https://github.com/go-modules-by-example/index/tree/master/012_modvendor

That particular example is showing an alternative to traditional vendoring, and I suspect that is not exactly what you'd want, but it serves as a concrete example of the types of things that are enabled via the flexibility of GOPROXY=file:///.

In any event, good chance GOPROXY=file:/// does not help based whatever your constraints are, but wanted to at least mention it in case it gives you an idea, etc.

@stapelberg

This comment has been minimized.

Copy link
Contributor

stapelberg commented Jan 18, 2019

resulting builds will not be as reproducible as ordinary module-enabled builds.

Debian has its own reproducibility efforts, see https://reproducible-builds.org/. The binaries we build are reproducible, and have been before Go modules :)

@stapelberg Is this in the context of what I think in the past might have been described as "more or less one giant GOPATH" for Debian packaging of Go libraries

Yes. This is the current model and we have no plans to change that.

I am not sure how much flexibility you have at package time vs. build time,

Changes at packaging-time take a long time to trickle down to all our volunteers. Build-time changes are preferable for this particular situation.

how that changes for a package vs. a binary,

No change.

the level of control around setting env variables

We can set arbitrary env variables.

or your ability to use an auxiliary directory that contains auxiliary data like go.mod files...

We can do that, but every step we need to set up introduces additional complexity, which makes builds slower and harder to understand/debug.

...but depending on your constraints, you might be able to do some variety of using GOPROXY=file:/// and/or temporarily setting GOPATH (which controls the location of GOPATH/pkg/mod cache location). That might not help, but here is a concrete example (from @myitcv):

Yeah, we currently already synthesize a GOPATH. From this discussion it seems like we need to synthesize all go.mod files ever referenced by the transitive closure as well. I have not yet looked into how easy or hard this would be to do.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment