Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: Let go mod download block proxy sites OR Export Go's Known Hostnames Matching Functionality #40405

Open
marwan-at-work opened this issue Jul 25, 2020 · 0 comments

Comments

@marwan-at-work
Copy link
Contributor

@marwan-at-work marwan-at-work commented Jul 25, 2020

This issue is a continuation of #31458

While in the original issue, I was mostly focusing on how the functionality is meant to be used, this issue is about what the Go command can do to help go proxies avoid that vulnerability.

Go treats certain hostnames such as github.com, bitbucket.org and others as "well known" and therefore Go bypasses the ?go-get=1 logic. This means, Go immediately knows the VCS Destination of an import path without having to ask the hostname for it through a meta tag.

The logic for this functionality lives here

When it comes to a private module, this is important because a "GET github.com/user/private?go-get=1" will return a 404 even if you send it Basic Authentication credentials through a ".netrc" file because this is a Web request that requires cookie authentication and not an API request.

This is the only way go proxies have been able to support private module fetching from such code hosting sites. So it's important to know that if Go changed its behavior on these well-known hostnames, then all go proxies that deal with private modules would break.

However, there's a problem: say you have a proxy running on proxy.com and say a malicious user runs a vanity import server at loop.net. Say the malicious user makes it so that loop.net/pkg?go-get=1 returns a meta tag with a mod value that points to proxy.com -- If proxy.com runs a go mod download loop.net/pkg@v1.2.3, it will trigger an infinite loop.

Therefore, the only way for a GOPROXY to avoid that vulnerability is by doing either of the following:

  1. Write de-duplication logic that single-flights similar requests which would stop the infinite loop from happening. This echos what @hyangah suggested in the original issue. However, this works well when you have a single instance of a proxy but when you horizontally scale your proxy, this becomes a more difficult feature to implement. You would need to not only implement distributed lock, you would also need to propagate the error from one replica to the remaining replicas waiting on the lock.

  2. Telling go mod download to never fetch from the proxy that triggered it by passing it the proxy.com hostname. For example: GONOPROXYHOSTS=proxy.com go mod download loop.net/pkg@v1.2.3 would trigger an error.

  3. Have the GOPROXY call GET loop.net/pkg?go-get=1 itself to parse a possible meta tag with that vulnerability and error out before calling go mod download.
    Point 3 above would work well for public modules but it would not work for private modules as I mentioned above that go-get=1 would not work on well known sites that have private modules such as github.com, bitbucket.org etc
    Therefore, the GOPROXY would have to check the import path and if it is a well-known hostname, then it wouldn't need to call "?go-get=1" at all because we know that Go itself wouldn't do it and therefore there is no vulnerability.

Personally, I believe option 2 is the easiest one from a proxy perspective though I can't speak for the Go implementation.

Thanks!

CC: @bcmills @jayconrod

@cagedmantis cagedmantis added this to the Backlog milestone Jul 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.