x/pkgsite: package proxy caching, removal request and legal concerns

[yordis]

I'm writing to address a critical issue involving the Go module proxy (proxy.golang.org) and pkg.go.dev caching packages from our monorepo, which is causing significant legal concerns for us.

First, I want to acknowledge that documentation exists on this topic, and I take full responsibility for any oversight on our part.
However, I hope you understand that the default behavior of automatically caching packages from public repositories is uncommon in other ecosystems and has led to unintended and problematic consequences in our case.

Context

Our project is a monorepo hosted on GitHub with multiple go.mod files for internal modules intended strictly for internal development, not for external distribution.
Although the repository was public on GitHub, I did not consider this as "publishing" the packages. The automatic caching of our internal modules occurred without our explicit consent, simply because the repository was accessible, leading to unintended public exposure.

That poses legal issues for our organization, which we are trying to resolve upon request of the lawyer at the moment to prevent future hassle.

Please advise on how we can proceed through a private and secure channel to provide necessary details without risking further exposure.

Concerns

I am concerned about the default behavior of the proxy automatically caching any publicly accessible repository without an opt-in mechanism or explicit consent. That creates significant legal and operational challenges, and the current process for removing cached packages is suboptimal.

While I am, now, aware of the GOPRIVATE environment variable, it is not practical for us.
It requires all contributors, including external freelancers using their own devices, to configure their environments appropriately, which isn't always enforceable. Unless I am misunderstanding the situation, I would appreciate your advise.

I believe that having an opt-in mechanism for caching packages—ideally specified in the go.mod file rather than an environment variable, or any other static file that could be committed. I am not sure.

I understand the motivation behind it, and I support it fully, my personal concern is having such behavior without manually opting for it.

Especially that the lack of a private communication channel to resolve this matter complicates the situation, as we are hesitant to discuss sensitive legal concerns in a public forum. I tried to find an email but I couldn't find anything other than GitHub issues.

[ianlancetaylor]

I'm sorry that you are having this trouble.

The GitHub (not Go) terms of service make it clear that if you post a repository publicly on GitHub, you are granting everyone the right to view and fork that repository. The Go module proxy is just one of many services that duplicate content hosted on GitHub. GitHub does offer alternatives like private repositories, and the Go module proxy will not duplicate private repositories; there are also private hosting services other than GitHub. If duplicating the code in your public GitHub repository is a problem for you, then you have problems beyond the Go module proxy.

That said, we can permanently remove packages from the Go module proxy. Please send complete details about the packages that should be removed to security@golang.org and reference this issue. We will of course want some sort of proof that you are the owner of the packages in question.

[hyangah]

Thanks @ianlancetaylor for the response.

I want to add one more:

Proxy.golang.org and pkg.go.dev cache/display modules and packages
only if they meet the following license requirement. https://pkg.go.dev/license-policy

(if you want to leave your private modules in public space but don't want proxy.golang.org/pkg.go.dev to cache, state your desired terms of use explicitly in your modules LICENSEs or completely skip having LICENSE files. That will prevent your modules from passing the license checker so proxy and pkgsite won't try to cache them).

To discuss the complete details or request permanent removals of accidentally published modules, please send your request to security@golang.org.

And see https://proxy.golang.org/ and https://pkg.go.dev/about for more details about the policy and communication channels.

Thanks.