Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: maintain a GOPATH-wide go.sum #24117

Closed
FiloSottile opened this issue Feb 25, 2018 · 10 comments

Comments

@FiloSottile
Copy link
Member

commented Feb 25, 2018

vgo keeps a global cache. In the same way it should keep a global go.modverify file, reminiscent of ssh's known_hosts.

This reduces the TOFU window of attack to just the first time a developer machine fetches a repository, which combined with the repository's go.modverify provides a strong trust waterfall.

When developers maintain multiple projects it will also make the risk of detection propagate to multiple communities, making targeted attacks even more different.

Finally, it could eventually be compared against and merged with any new go.modverify encountered, making verification automatic and making trust also flow from users back to the developer.

Aside from security, it will also provide more pressure against changing tag values, because it will be harder to remove a pin.

While the cache might need size management and/or expiration features, the global go.modverify should not reach meaningful size issues, preventing eviction attack and providing permanent pinning. (And maybe the base for future gossiping features.)

@gopherbot gopherbot added this to the vgo milestone Feb 25, 2018

@sdboyer

This comment has been minimized.

Copy link
Member

commented Feb 25, 2018

Strong agree - the class of information being kept in go.modverify makes more sense in some kind of central location, rather than being split up across projects.

@FiloSottile

This comment has been minimized.

Copy link
Member Author

commented Feb 25, 2018

To clarify, I think it's important for it to be both in the repository (#24116) and in a global location, possibly with cross-pollination.

The go.modverify in the repository is critical to ensure users have only a single well-understood point of security failure: fetching the source.

@airylinus

This comment has been minimized.

Copy link

commented Mar 22, 2018

I agree with that.
For instance, package golang.org/x/net actually repo locates is github.com/golang/net. This is pretty creepy when you clone a new repo and stuck at a go build error.

@rsc

This comment has been minimized.

Copy link
Contributor

commented Mar 30, 2018

Not yet. We're fixing one thing at a time. The first thing to fix is management of versions at all. The second thing is verification. There's no need to do both at once. We've gotten by this long with "go get" with no modverify. Let's get versions into go first, and then turn our attention to verifying.

@rsc

This comment has been minimized.

Copy link
Contributor

commented Apr 9, 2018

Brad asked me to kill uses of NeedsFixButNotYet, so I'm going to call this NeedsDecision.

@rsc rsc modified the milestones: vgo, vgo2 Jun 6, 2018

@rsc rsc modified the milestones: vgo2, Go1.12 Jul 12, 2018

@rsc rsc changed the title x/vgo: maintain a global go.modverify cmd/go: maintain a global go.modverify Jul 12, 2018

@rsc rsc added the modules label Jul 12, 2018

@FiloSottile FiloSottile changed the title cmd/go: maintain a global go.modverify cmd/go: maintain a global go.sum Aug 16, 2018

@bcmills bcmills modified the milestones: Go1.12, Go1.13 Nov 14, 2018

@bcmills

This comment has been minimized.

Copy link
Member

commented Nov 14, 2018

I have a (non-global) counterproposal in #28802. I'll follow up with @FiloSottile to try to better understand how the differences play out under various possible threat models.

@rsc

This comment has been minimized.

Copy link
Contributor

commented Feb 26, 2019

Re @bcmills's comment, #28802 would probably help but is not a complete solution. For example, it does not address go get -u. This issue is about a complete solution.

@rsc

This comment has been minimized.

Copy link
Contributor

commented Mar 4, 2019

@FiloSottile explained to me off-issue that "global" here really meant "GOPATH-wide".
(Note the mention of "global cache".)

@rsc rsc changed the title cmd/go: maintain a global go.sum cmd/go: maintain a GOPATH-wide go.sum Mar 4, 2019

@rsc

This comment has been minimized.

Copy link
Contributor

commented Mar 4, 2019

For what it's worth, I think a GOPATH-wide go.sum is a mistake. It introduces a new not-automatically-managed piece of state into what is supposed to be a stateless cache. We've worked hard to eliminate state and state manipulations (like needing to run "go install" to populate the stateful cache that is GOPATH/pkg). The notary should take care of the underlying need here.

The only override for a problem here would be to edit the GOPATH-wide go.sum, which is what I mean by having a not-automatically-managed piece of state. Now the cache is not a cache.

For a similar reason I think we should probably not do #28802: recovering from a problem is too hard and depends on manipulating state outside the current module.

@rsc

This comment has been minimized.

Copy link
Contributor

commented Apr 30, 2019

The sumdb support caches results from the checksum database but also reauthenticates them before use. That has the effect of maintaining a GOPATH-wide go.sum but avoids the state problems I mentioned in the last comment. I'm going to close this as addressed by use of the checksum database with this caching (and otherwise not something we want to do).

@rsc rsc closed this Apr 30, 2019

harmony-ek added a commit to harmony-ek/harmony that referenced this issue May 14, 2019
Add go.sum back
To quote the “Releasing Modules (All Version)” section of the Go Modules
wiki page [1]:

    Ensure your go.sum file is committed along with your go.mod file.
    See FAQ below [2] for more details and rationale.

And the “Should I commit my 'go.sum' file as well as my 'go.mod' file?”
section from the same page [2]:

    Typically your module's go.sum file should be committed along with your
    go.mod file.

      - go.sum contains the expected cryptographic checksums of the
        content of specific module versions.

      - If someone clones your repository and downloads your
        dependencies using the go command, they will receive an error if
        there is any mismatch between their downloaded copies of your
        dependencies and the corresponding entries in your go.sum.

      - In addition, go mod verify checks that the on-disk cached copies
        of module downloads still match the entries in go.sum.

      - Note that go.sum is not a lock file as used in some alternative
        dependency management systems. (go.mod provides enough
        information for reproducible builds).

      - See very brief rationale here [3] from Filippo Valsorda on why
        you should check in your go.sum. See the "Module downloading and
        verification" [4] section of the tip documentation for more
        details.  See possible future extensions being discussed for
        example in golang/go#24117 and golang/go#25530.”

[1] https://github.com/golang/go/wiki/Modules#releasing-modules-all-versions
[2] https://github.com/golang/go/wiki/Modules#should-i-commit-my-gosum-file-as-well-as-my-gomod-file
[3] https://twitter.com/FiloSottile/status/1029404663358087173
[4] https://tip.golang.org/cmd/go/#hdr-Module_downloading_and_verification
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.