proposal: cmd/go: allow GOPRIVATE to provide source repository URI #45611
Proposal: Allow GOPRIVATE to provide source repository URI
Last updated: 2021/04/24
In current state of Go modules' tooling, working with private repositories without setting up a private proxy infrastructure serving modules over https is possible but cumbersome; and still is confined to git VCS only.
For git repositories it is usually done like that:
go env -w GOPRIVATE=example.com git config --global url."email@example.com:".insteadOf "https://example.com"
This configuration works, but it has to be global (per machine/user). To my knowledge no other VCS allows for such workaround.
#40189 : add a sub command to manage authentication
Allow GOPRIVATE to point to the source repository for a given module path matching pattern. Let user provide vcs type, transport, and credentials.
GOPRIVATE=example.com/[fossil+ssh://internal.example.com] GOPRIVATE=example.com/[git+ssh://firstname.lastname@example.org] GOPRIVATE=example.com/[fossil+https://internal.example.com] GOPRIVATE=example.com/[git+https://testing.example.com/test]
If pattern ends with slash, allow URI to be constructed using parts of matched module path represented by five percent-letter variables:
Reserved characters of RFC 3986 that should be literal part of produced URI must be given in percent-encoding form. Literal characters ":", "[", "]", "", and "@" can also be provided after a backslash character.
The text was updated successfully, but these errors were encountered:
Note to the readers: (answering doubts raised at irc discussion) This proposal is just about simplifying private repository setup for solo developers and small teams. As per docs GOPRIVATE affects GONOPROXY and GONOSUMDB. Ie. modules still are "downloaded" (here copied) to the GOMODCACHE as if coming from any other source.
This proposal is entirely separate to #44347.
Also, the "keep GOPATH for private repositories" attitude will better be served by #44649 proposal.
@ohir I think #40189 and this issue are two different concerns. This issue seems to be about making GOPRIVATE be smarter when authenticating with a VCS backend. While #40189 is about having a sub command that manages authenticating with a GOPROXY.
Although I agree that authenticating with private Go Modules has always been a point of friction, the syntax here seems a bit overloaded and potentially even harder for people to configure correctly.
Go, keeping a small surface area in how it interacts with VCS systems like
I think for that reason, a GOPROXY is a better answer which means only one server has to configure their VCS credentials correctly and everybody else wouldn't even need to have a VCS program installed on their machine to fetch private Go Modules.
However, the current state of things means that the authentication mechanism with a GOPROXY is almost as cumbersome as the authentication mechanism with a VCS sub system, hence the reason for #40189
This is pretty much (except for vcs+ part) syntax used by most CLI net tools (wget, curl, ftp, even git itself). Anyway, I got an answer I asked for, thank you :)
GOPRIVATE is really for encoding one specific piece of data. Trying to add more data to the encoding does not seem like it is the right approach. It seems like there's a more general problem here to solve, but I'm not sure what the exact problem statement is and how much to generalize it.
This specific piece of data is enough to opt-out of public, but not enough to work in private.
Setting up private repositories is cumbersome for small enterprises and for solo developers. Going private needs either a fragile and error-prone configuration dispersed among at least three env variables (GOPRIVATE GOPROXY GONOPROXY), and possibly local vcs configuration — or it needs a private goproxy server like Athens to be configured and accessible.
This proposal conceives that GOPRIVATE should be the single place where small teams and solo developers can setup all things "private" — without need to think about GOPROXY order and providing external authentication tools (as proposed in #40189).
Then tools can access vcs repository and add missing private modules to GOMODCACHE /cache/downloads obeying goproxy protocol.
It's clear we need an answer for the underlying problem that both this issue and #40189 are trying to solve, but neither really solves it completely. We don't have a coherent plan right now. This is something the go command team hopes to look at in the next cycle. Putting on hold until then.