Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: GOPROXY should not override GONOPROXY/GOPRIVATE #56477

Closed
FiloSottile opened this issue Oct 28, 2022 · 8 comments
Closed

cmd/go: GOPROXY should not override GONOPROXY/GOPRIVATE #56477

FiloSottile opened this issue Oct 28, 2022 · 8 comments
Labels
GoCommand cmd/go modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone

Comments

@FiloSottile
Copy link
Contributor

FiloSottile commented Oct 28, 2022

What version of Go are you using (go version)?

$ go version
go version go1.19.2 darwin/arm64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="arm64"
GOBIN=""
GOCACHE="/Users/filippo/Library/Caches/go-build"
GOENV="/Users/filippo/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/filippo/pkg/mod"
GONOPROXY="github.com/FiloSottile/*,filippo.io/*"
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/filippo"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org"
GOROOT="/opt/homebrew/Cellar/go/1.19.2/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/opt/homebrew/Cellar/go/1.19.2/libexec/pkg/tool/darwin_arm64"
GOVCS=""
GOVERSION="go1.19.2"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/filippo/src/filippo.io/mostly-harmless/filippo.fly.dev/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/var/folders/_j/hq4ytn1n4b94fhrpvvb9tktr0000gn/T/go-build3821845917=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

$ GOPRIVATE="c2sp.org" go mod download -json -x c2sp.org/CCTV/age@main

What did you expect to see?

A direct connection to c2sp.org.

What did you see instead?

# get https://proxy.golang.org/c2sp.org/%21c%21c%21t%21v/age/@v/main.info
# get https://proxy.golang.org/c2sp.org/%21c%21c%21t%21v/age/@v/main.info: 404 Not Found (5.486s)
{
	"Path": "c2sp.org/CCTV/age",
	"Version": "main",
	"Error": "c2sp.org/CCTV/age@main: invalid version: reading https://proxy.golang.org/c2sp.org/%21c%21c%21t%21v/age/@v/main.info: 404 Not Found\n\tserver response: not found: c2sp.org/CCTV/age@main: unrecognized import path \"c2sp.org/CCTV/age\": https://c2sp.org/CCTV/age?go-get=1 and //c2sp.org/CCTV?go-get=1 disagree about go-import for c2sp.org/CCTV"
}

Ignore the error, the point is that with GOPRIVATE set this request should not have reached the proxy.

@heschi heschi added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Oct 28, 2022
@heschi
Copy link
Contributor

heschi commented Oct 28, 2022

cc @bcmills @matloob

@heschi heschi added this to the Go1.20 milestone Oct 28, 2022
@seankhliao
Copy link
Member

seankhliao commented Oct 28, 2022

has your GOPROXY been explicitly set? it's non default and I believe that overrides GOPRIVATE (to allow you to use private proxies).

@heschi
Copy link
Contributor

heschi commented Oct 28, 2022

Oh, good catch. Though I think you meant GONOPROXY.

@FiloSottile
Copy link
Contributor Author

FiloSottile commented Oct 28, 2022

Oh yeah, good catch. I do that to mitigate VCS vulnerabilities. However, I would expect using GONOPROXY without direct in GOPROXY to be an error at most, not to silently ignore GONOPROXY.

https://go.dev/ref/mod#private-module-privacy explicitly says (emphasis mine)

The GOPRIVATE or GONOPROXY environment variables may be set to lists of glob patterns matching module prefixes that are private and should not be requested from any proxy.

and then it goes on to explain not to set GONOPROXY if you're using a private proxy.

@FiloSottile FiloSottile changed the title cmd/go: GOPRIVATE didn't apply unexpectedly cmd/go: GOPROXY should not override GONOPROXY/GOPRIVATE Oct 28, 2022
@seankhliao
Copy link
Member

seankhliao commented Oct 28, 2022

I think heschi is correct in saying it's your GONOPROXY setting that's overriding it.

https://go.dev/ref/mod#private-modules :

GOPRIVATE — list of glob patterns of module path prefixes that should be considered private. Acts as a default value for GONOPROXY and GONOSUMDB.

with an explicit GONOPROXY="github.com/FiloSottile/*,filippo.io/*", the default value from GOPRIVATE isn't used.

@bcmills
Copy link
Member

bcmills commented Oct 28, 2022

@FiloSottile

https://go.dev/ref/mod#private-module-privacy explicitly says (emphasis mine)

You left out the very next sentence, which is key to the behaviors you observe here:

GOPRIVATE simply acts as a default for GONOPROXY and GONOSUMDB, so it’s not necessary to set GONOPROXY unless GONOSUMDB should have a different value.

That also matches https://go.dev/ref/mod#environment-variables:

Variable Description
GOPRIVATE Comma-separated list of glob patterns (in the syntax of Go's path.Match) of module path prefixes that should be considered private. GOPRIVATE is a default value for GONOPROXY and GONOSUMDB. See Privacy. GOPRIVATE also determines whether a module is considered private for GOVCS.

Because you have set GONOPROXY explicitly in the environment, that overrides the default set by GOPRIVATE.
Because c2sp.org does not match a pattern in GONOPROXY, it is fetched using your GOPROXY setting.

(The GOPRIVATE setting still affects GONOSUMDB and the interpretation of private: in GOVCS.)

@bcmills bcmills modified the milestones: Go1.20, Backlog Oct 28, 2022
@bcmills
Copy link
Member

bcmills commented Oct 28, 2022

As far as I can tell this is working as documented. It's unfortunate that the environment variables are this complex, but on the other hand I don't think most users are setting GONOPROXY at all.

@FiloSottile, do you want to file a proposal for a behavior change here? Is there something we could do to clarify the documentation? (I'd be happy to review a CL to clean up the docs.)

@bcmills bcmills added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Oct 28, 2022
@gopherbot
Copy link

gopherbot commented Nov 28, 2022

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
GoCommand cmd/go modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Development

No branches or pull requests

5 participants