Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: "unknown revision" error due to git client corruption in the Go module cache #43260

Open
dzrtc opened this issue Dec 18, 2020 · 5 comments
Open

Comments

@dzrtc
Copy link

@dzrtc dzrtc commented Dec 18, 2020

What version of Go are you using (go version)?

go version go1.14.2 darwin/amd64

Does this issue reproduce with the latest release?

I don't know how to repro this issue; it appears intermittently.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env

What did you do?

I created a new private repo on GitHub, populated it, and about ~10 minutes later attempted to go get it into another project on my local machine. This failed, and after several additional commands (below), I realized that the mod cache entry itself was corrupted.

Manually deleting the entry from the mod cache, then issuing the original command (command 1 below) again appears to have resolved the issue.

What did you expect to see?

That the module downloaded.

What did you see instead?

Below:

Command 1

GOPRIVATE=github.com/xxxx GIT_TERMINAL_PROMPT=1 go get -u github.com/xxxx/yyyy@master
go get github.com/xxxx/yyyy@master: github.com/xxxx/yyyy@master: invalid version: unknown revision master

Command 2

Same as command 1, yet without @master.

GOPRIVATE=github.com/xxxx GIT_TERMINAL_PROMPT=1 go get -u github.com/xxxx/yyyy
go get github.com/xxxx/yyyy: module github.com/xxxx/yyyy: git ls-remote -q origin in /Users/zzzzz/go/pkg/mod/cache/vcs/916ad3826f1a995d5f8341487e6563a6c380c2d8aef514d8034f962111c7b16c: exit status 128:
	fatal: 'origin' does not appear to be a git repository
	fatal: Could not read from remote repository.

	Please make sure you have the correct access rights
	and the repository exists.

Command 3

Same as command 2, yet without GOPRIVATE, GIT_TERMINAL_PROMPT.

go get github.com/xxxx/yyyy: module github.com/xxxx/yyyy: git ls-remote -q origin in /Users/zzzzz/go/pkg/mod/cache/vcs/916ad3826f1a995d5f8341487e6563a6c380c2d8aef514d8034f962111c7b16c: exit status 128:
	fatal: 'origin' does not appear to be a git repository
	fatal: Could not read from remote repository.

	Please make sure you have the correct access rights
	and the repository exists.

Mitigation

After trying a number of things, I decided to just manually purge the entry from the cache.

cd $HOME/go/pkg/mod/cache/vcs
rm 916ad3826f1a995d5f8341487e6563a6c380c2d8aef514d8034f962111c7b16c.info
rm 916ad3826f1a995d5f8341487e6563a6c380c2d8aef514d8034f962111c7b16c.lock
rm -rf ./916ad3826f1a995d5f8341487e6563a6c380c2d8aef514d8034f962111c7b16c/

After that, reissuing command 1 (above), resulted in the module being downloaded successfully as originally expected. Note that both GOPRIVATE and @master are present in this successful attempt.

GOPRIVATE=github.com/xxxx GIT_TERMINAL_PROMPT=1 go get -u github.com/xxxx/zzzz@master
go: downloading github.com/xxxx/yyyy v0.0.0-20201218021721-ab6f5cd5f994
go: github.com/xxxx/yyyy master => v0.0.0-20201218021721-ab6f5cd5f994
go: golang.org/x/sys upgrade => v0.0.0-20201214210602-f9fddec55a1e
go: golang.org/x/crypto upgrade => v0.0.0-20201217014255-9d1352758620
go: github.com/pkg/errors upgrade => v0.9.1
go: github.com/sirupsen/logrus upgrade => v1.7.0

Discussion

The documentation strongly communicates that the mod cache is immutable and shouldn't be touched. The tooling reinforces that by treating the mod cache as an all-or-nothing construct -- go clean --modcache drops the entire thing.

However, with this issue and mitigation, I've learned that individual entries get corrupted in a way that can be corrected by surgically deleting a specific cache entry.

It would be extremely helpful if the the go clean command were to be updated to allow an individual module to be purged from the mod cache (and for the various go get error paths and/or documentation to suggest that purging a single module from the cache can resolve issues).

@dzrtc dzrtc changed the title cmd/go: stale mod cache entries must be removed by hand cmd/go: broken mod cache entries must be removed by hand Dec 18, 2020
@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Dec 21, 2020

#32976 is a duplicate issue for removing specific modules from the module cache.

I'm much more concerned about cache corruption. Can you give us any more information on how to reproduce this? Does this happen on more recent versions?

cc @bcmills @matloob

@bcmills
Copy link
Member

@bcmills bcmills commented Dec 21, 2020

The unknown revision error generally occurs before anything is written to the module cache proper. The similarity to #34092 makes me suspect a git client bug — @dzrtc, with what version of git did you observe this error?

@dzrtc
Copy link
Author

@dzrtc dzrtc commented Dec 22, 2020

git version 2.24.1

@bcmills
Copy link
Member

@bcmills bcmills commented Dec 22, 2020

@dzrtc, thanks. That's a new enough version that we ought to be able to support it. It appears that the --unshallow bug is still not fixed upstream, but we already have a workaround in place for that one with the go command.

So regardless of whether this is a go bug or a git bug, we will unfortunately need more information to go on. If you see this issue again, could you write down (as best as you can recall) the steps leading up to it, and perhaps save a copy of the git repo from your Go module cache (or the entire module cache if you prefer) so that we can analyze what's wrong with it?

@bcmills bcmills added this to the Unplanned milestone Dec 22, 2020
@bcmills bcmills changed the title cmd/go: broken mod cache entries must be removed by hand cmd/go: "unknown revision" error due to git client corruption in the Go module cache Dec 22, 2020
@dzrtc
Copy link
Author

@dzrtc dzrtc commented Dec 22, 2020

Yes, I should've saved the modcache repo; I'll watch for it to reoccur and try to save more state.

I have no particular dependency on 2.24, and could upgrade.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.