Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: support fetching git repos with sha256 hashes #68359

Open
johnnybubonic opened this issue Jul 9, 2024 · 3 comments
Open

cmd/go: support fetching git repos with sha256 hashes #68359

johnnybubonic opened this issue Jul 9, 2024 · 3 comments
Labels
FeatureRequest GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.

Comments

@johnnybubonic
Copy link

johnnybubonic commented Jul 9, 2024

Go version

go version go1.22.4 linux/amd64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/[REDACTED USER]/.cache/go-build'
GOENV='/home/[REDACTED USER]/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/opt/dev/go/pkg/mod'
GONOPROXY='[REDACTED REPO]/[REDACTED NAME],[REDACTED REPO]/[REDACTED ORG]/*,github.com/[REDACTED ORG2]/*'
GONOSUMDB='[REDACTED REPO]/[REDACTED NAME],[REDACTED REPO]/[REDACTED ORG]/*,github.com/[REDACTED ORG2]/*'
GOOS='linux'
GOPATH='/opt/dev/go'
GOPRIVATE='[REDACTED REPO]/[REDACTED NAME],[REDACTED REPO]/[REDACTED ORG]/*,github.com/[REDACTED ORG2]/*'
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/lib/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/lib/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.22.4'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build1827441876=/tmp/go-build -gno-record-gcc-switches'

What did you do?

Let me preface this that this is likely (also?) an issue with the Go mod proxy, and my apologies if there's already an open issue for this but I looked and didn't see one (though one can imagine searching for "git" "sha256" in the Issues is going to return a lot of unrelated issues regardless for this project).

It is, however, unfortunately painfully apparent that SHA256 commit hashes do not seem to be supported:

(NOTE: Please see this comment; the module shown below has been recreated with SHA1 hashes and a new repository/module has been created to test with.)

$ go mod download r00t2.io/wireproto@v1.0.0
go: r00t2.io/wireproto@v1.0.0: invalid version: git fetch -f origin refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in /opt/dev/go/pkg/mod/cache/vcs/3e171304ac1e55b26a8f8d1dae1cb29af5fc6089d97d912ba5cb27873649d225: exit status 128:
	fatal: Server does not support this repository's object format
	fatal: the remote end hung up unexpectedly
	Gitea: Failed to execute git command

See also here, which provides a bit more useful info.

The same issue is present with 1.23 RC1 as well, of course:

$ go1.23rc1 mod download r00t2.io/wireproto@v1.0.0
go: r00t2.io/wireproto@v1.0.0: invalid version: git fetch -f origin refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in /opt/dev/go/pkg/mod/cache/vcs/3e171304ac1e55b26a8f8d1dae1cb29af5fc6089d97d912ba5cb27873649d225: exit status 128:
	fatal: Server does not support this repository's object format
	fatal: the remote end hung up unexpectedly
	Gitea: Failed to execute git command

The repository itself is fine (I have so-called "vanity" Go module roots/import paths):

$ git clone git@git.r00t2.io:r00t2/go_wireproto && echo '##' ; rm -rf go_wireproto ; git clone https://git.r00t2.io/r00t2/go_wireproto && echo '##' ; cd go_wireproto && git tag -l
Cloning into 'go_wireproto'...
remote: Enumerating objects: 74, done.
remote: Counting objects: 100% (21/21), done.
remote: Compressing objects: 100% (21/21), done.
remote: Total 74 (delta 7), reused 0 (delta 0), pack-reused 53 (from 1)
Receiving objects: 100% (74/74), 61.25 KiB | 1.80 MiB/s, done.
Resolving deltas: 100% (31/31), done.
##
Cloning into 'go_wireproto'...
remote: Enumerating objects: 74, done.
remote: Counting objects: 100% (21/21), done.
remote: Compressing objects: 100% (21/21), done.
remote: Total 74 (delta 7), reused 0 (delta 0), pack-reused 53 (from 1)
Receiving objects: 100% (74/74), 61.25 KiB | 1.53 MiB/s, done.
Resolving deltas: 100% (31/31), done.
##
v1.0.0

The redirect is correct:

$ curl -H "user-agent: Go-http-client/v1" -sL "https://r00t2.io/wireproto?go-get=1" | grep -F 'content='
		<meta name="go-import" content="r00t2.io/wireproto git https://git.r00t2.io/r00t2/go_wireproto">

And it works fine with older repositories still using SHA1 commit hashes.

What did you see happen?

(See above)

What did you expect to see?

SHA256 commit hashes supported (as they've been in upstream git since ~ early 2021-ish?).

@seankhliao
Copy link
Member

The error form the .info endpoint:

not found: r00t2.io/wireproto@v1.0.0: invalid version: git fetch -f origin refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in /tmp/gopath/pkg/mod/cache/vcs/3e171304ac1e55b26a8f8d1dae1cb29af5fc6089d97d912ba5cb27873649d225: exit status 128:
	fatal: mismatched algorithms: client sha1; server sha256

is the same as if you used GOPROXY=direct:

$ go get -v -x r00t2.io/wireproto@v1.0.0
# get https://r00t2.io/?go-get=1
# get https://r00t2.io/wireproto?go-get=1
# get https://r00t2.io/?go-get=1: 200 OK (0.775s)
# get https://r00t2.io/wireproto?go-get=1: 200 OK (0.882s)
get "r00t2.io/wireproto": found meta tag vcs.metaImport{Prefix:"r00t2.io/wireproto", VCS:"git", RepoRoot:"https://git.r00t2.io/r00t2/go_wireproto"} at //r00t2.io/wireproto?go-get=1
mkdir -p /home/user/.data/go/pkg/mod/cache/vcs # git3 https://git.r00t2.io/r00t2/go_wireproto
# lock /home/user/.data/go/pkg/mod/cache/vcs/3e171304ac1e55b26a8f8d1dae1cb29af5fc6089d97d912ba5cb27873649d225.lock
# /home/user/.data/go/pkg/mod/cache/vcs/3e171304ac1e55b26a8f8d1dae1cb29af5fc6089d97d912ba5cb27873649d225 for git3 https://git.r00t2.io/r00t2/go_wireproto
cd /home/user/.data/go/pkg/mod/cache/vcs/3e171304ac1e55b26a8f8d1dae1cb29af5fc6089d97d912ba5cb27873649d225; git tag -l
0.004s # cd /home/user/.data/go/pkg/mod/cache/vcs/3e171304ac1e55b26a8f8d1dae1cb29af5fc6089d97d912ba5cb27873649d225; git tag -l
cd /home/user/.data/go/pkg/mod/cache/vcs/3e171304ac1e55b26a8f8d1dae1cb29af5fc6089d97d912ba5cb27873649d225; git ls-remote -q origin
0.519s # cd /home/user/.data/go/pkg/mod/cache/vcs/3e171304ac1e55b26a8f8d1dae1cb29af5fc6089d97d912ba5cb27873649d225; git ls-remote -q origin
cd /home/user/.data/go/pkg/mod/cache/vcs/3e171304ac1e55b26a8f8d1dae1cb29af5fc6089d97d912ba5cb27873649d225; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' 15fd8df0bff4b9b0f2c455422e57ef2bc6f8add5c95e045f9b51021874f22440 --
0.004s # cd /home/user/.data/go/pkg/mod/cache/vcs/3e171304ac1e55b26a8f8d1dae1cb29af5fc6089d97d912ba5cb27873649d225; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' 15fd8df0bff4b9b0f2c455422e57ef2bc6f8add5c95e045f9b51021874f22440 --
cd /home/user/.data/go/pkg/mod/cache/vcs/3e171304ac1e55b26a8f8d1dae1cb29af5fc6089d97d912ba5cb27873649d225; git -c protocol.version=2 fetch -f --depth=1 origin refs/tags/v1.0.0:refs/tags/v1.0.0
0.513s # cd /home/user/.data/go/pkg/mod/cache/vcs/3e171304ac1e55b26a8f8d1dae1cb29af5fc6089d97d912ba5cb27873649d225; git -c protocol.version=2 fetch -f --depth=1 origin refs/tags/v1.0.0:refs/tags/v1.0.0
cd /home/user/.data/go/pkg/mod/cache/vcs/3e171304ac1e55b26a8f8d1dae1cb29af5fc6089d97d912ba5cb27873649d225; git fetch -f origin 'refs/heads/*:refs/heads/*' 'refs/tags/*:refs/tags/*'
0.504s # cd /home/user/.data/go/pkg/mod/cache/vcs/3e171304ac1e55b26a8f8d1dae1cb29af5fc6089d97d912ba5cb27873649d225; git fetch -f origin 'refs/heads/*:refs/heads/*' 'refs/tags/*:refs/tags/*'
go: r00t2.io/wireproto@v1.0.0: invalid version: git fetch -f origin refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in /home/user/.data/go/pkg/mod/cache/vcs/3e171304ac1e55b26a8f8d1dae1cb29af5fc6089d97d912ba5cb27873649d225: exit status 128:
	fatal: mismatched algorithms: client sha1; server sha256

go fetches the a version using git init + git remote add + fetches,
and the local init will use sha1, resulting in a mismatch with the remote using sha256

https://go.googlesource.com/go/+/e89e880eacb1aceaa14733f44c38e6748d5ffa9c/src/cmd/go/internal/modfetch/codehost/git.go#83

@seankhliao seankhliao changed the title cmd./go: go mod (nor pkg.go.dev) doesn't support SHA256 git repos cmd/go: support fetching git repos with sha256 hashes Jul 9, 2024
@seankhliao seankhliao added FeatureRequest GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Jul 9, 2024
@johnnybubonic
Copy link
Author

I needed this library accessible for other projects, so I have deleted the above repository and re-recreated it using SHA1 commit hashes.

In its place, I have turned up a testing repository/module that can be used (and is quite a bit smaller) to aid in testing SHA256 commit hashes with go mod:

https://git.r00t2.io/r00t2/go_sha256repo

The import path is r00t2.io/sha256repo; I have set up an identical import URL for it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
FeatureRequest GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

3 participants