Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: cmd/go: allow users to control VCS usage #41730

Open
rsc opened this issue Oct 1, 2020 · 5 comments
Open

proposal: cmd/go: allow users to control VCS usage #41730

rsc opened this issue Oct 1, 2020 · 5 comments
Labels
Projects
Milestone

Comments

@rsc
Copy link
Contributor

@rsc rsc commented Oct 1, 2020

The go command runs commands like git and hg to download modules. In the past, we have had problems with security bugs in version control systems becoming security bugs in “go get”.

The original modules draft design removed use of these commands entirely, saying:

We want to move away from invoking version control tools such as bzr, fossil, git, hg, and svn to download source code. These fragment the ecosystem: packages developed using Bazaar or Fossil, for example, are effectively unavailable to users who cannot or choose not to install these tools. The version control tools have also been a source of exciting security problems. It would be good to move them outside the security perimeter.

The removal of these commands was not possible in the end: being able to fetch directly from Git repos is too important, especially for closed source. But the security exposure has not gone away. We remain vulnerable to problems in VCS systems, especially the less scrutinized ones.

I propose to add a new GOVCS setting that controls which version control systems are allowed to be used for downloads.

In its simplest form, GOVCS is a pipe-separated list of allowed version control systems:

GOVCS=bzr|fossil|git|hg|svn  # any known VCS system

The set of all known version control systems can be abbreviated “all’:

GOVCS=all  # any known VCS system

The set of no VCS systems can be written “off”:

GOVCS=off  # no VCS usage allowed at all

A glob pattern (already used for GOPRIVATE, GONOPROXY, and so on) followed by a colon can restrict the effect to a particular (module or import) path prefix:

GOVCS=github.com:git  # on github.com, only use git

The special pattern “private” means any pattern from GOPRIVATE.
The special pattern “public” means anything not matched by GOPRIVATE.

A comma-separated list of restricted can be given. The first match in the list applies:

GOVCS=github.com:git,bitbucket.com:git|hg

Finally, there is a default setting in the go command itself. An explicit setting of GOVCS takes priority but does not remove the defaults; the effect is as if the explicit GOVCS setting were inserted ahead of the defaults in the comma-separated list.

(Note that this gives considerably more control over exactly when a VCS can be used than the current behavior of “if it’s in the PATH, it can be used.”)

I propose the default GOVCS setting allow all VCS for private modules but only Git and Mercurial for public modules.

That is, the default setting is equivalent to:

GOVCS=private:all,public:git|hg

The rationale behind allowing only Git and Mercurial is that these two systems have had the most attention to issues of being run as clients of untrusted servers. In contrast, Bazaar, Fossil, and Subversion have primarily been used in trusted, authenticated environments and are not as well scrutinized as attack surfaces.

It is certainly the case that even Git and Mercurial are unfortunately not free of bugs either, but those projects have had more scrutiny and have established a record of fixing them quickly.

The rationale behind allowing private:all is that private servers are trusted. It is beyond the go command’s threat model to guard against scenarios in which a company’s private source code host has already been compromised.

Users who want to allow all VCS even for public source code servers would use

GOVCS=private:all,public:all

I propose that the public Go module mirror proxy.golang.org continue to allow all VCS.

It is already the case that the Go module mirror handles fetching Bazaar, Fossil, and Subversion repos for the vast majority of Go users: those users do not even need to install the corresponding binaries, and it is likely that most do not.

The module mirror also invokes the go command and its subprocesses in a security sandbox for added security, so it is more equipped to deal with potential compromises of the version control tools. It would simply run with GOVCS=public:all to continue allowing all the VCS tools to be used. We would encourage alternate module mirrors to do the same.

The combination of disallowing less-popular version control systems by default but continuing to be able to fetch them through module mirrors balances improving baseline security on developer systems machines while not fragmenting the ecosystem.

This default does put the module mirror on the critical path for downloads of public Bazaar/Fossil/Subversion downloads, but it is the same system that serves the checksum database, which is already on the critical path for all downloads. And the setting that opts out of the checksum database and proxy for all modules—GOPRIVATE=*—would automatically opt out of the VCS restrictions as well.

@rsc rsc added the Proposal label Oct 1, 2020
@rsc rsc added this to the Proposal milestone Oct 1, 2020
@rsc rsc added this to Incoming in Proposals Oct 1, 2020
@FiloSottile
Copy link
Member

@FiloSottile FiloSottile commented Oct 2, 2020

Strong support. This would significantly reduce the attack surface of "go get".

I think the proposal could make the relative priority of comma and pipe more clear, but it looks like there is only one reasonable interpretation.

Once this is implemented, I think we should consider vulnerabilities in any VCS other than Git and Mercurial as out of scope for golang.org/security.

/cc @golang/security

@beoran
Copy link

@beoran beoran commented Oct 3, 2020

This is a step in the right direction, however I agree with the original "We want to move away from invoking version control tools".

In other programming languages, like say, Ruby, the equivalent of go modules is distributed as special purpose archives, like .gem files for Ruby. Go should allow something similar with a go module pack command that packs your module foo v1.2.3 to such an archive, maybe named foo@1.2.3.nut, which go get then can download over https, based on the go.mod file.

@bcmills
Copy link
Member

@bcmills bcmills commented Oct 5, 2020

@beoran, we already support a protocol that does not need to invoke any version control tools. The go-import metadata can indicate mod as the VCS, which causes the go command to use the GOPROXY protocol over native HTTPS.

I presume that under this proposal, GOVCS=off would still allow use of the mod protocol, since that does not require any tools other than the go command itself.

@rsc rsc moved this from Incoming to Active in Proposals Oct 7, 2020
@rsc
Copy link
Contributor Author

@rsc rsc commented Oct 14, 2020

It seems like everyone is in favor of this. Do I have that right? Are there any objections?

@beoran
Copy link

@beoran beoran commented Oct 14, 2020

@bcmills That is nice to know, but not very practical. Ruby gems are far easier to use than a raw goprogy. But I support this proposal as a step in the right direction.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.