Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: go get -d -u on AWS code commit repository fails #32166

Open
Baxtex opened this issue May 21, 2019 · 6 comments

Comments

Projects
None yet
5 participants
@Baxtex
Copy link

commented May 21, 2019

What version of Go are you using (go version)?

$ go version 12.5

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/CORPUSERS/xxx/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/CORPUSERS/xxx/projects/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=

What did you do?

Trying to run go get on a repository hosted on code commit:

go get -d -u git-codecommit.eu-west-1.amazonaws.com/v1/repos/myRepo

What did you expect to see?

The repository cloned and dependencies installed.

What did you see instead?

package git-codecommit.eu-west-1.amazonaws.com/v1/repos/myRepo: unrecognized import path "git-codecommit.eu-west-1.amazonaws.com/v1/repos/myRepo" (parse https://git-codecommit.eu-west-1.amazonaws.com/v1/repos/myRepo?go-get=1: no go-import meta tags ())

My machine is properly setup with SSH keys so git clone ssh://git-codecommit.eu-west-1.amazonaws.com/v1/repos/myRepo works fine. Also take note that my end goal is to the the build pipeline on AWS Code build run the dockerfile in the end but as of now it gives the same result as running it locally.

@AlexRouSg

This comment has been minimized.

Copy link
Contributor

commented May 21, 2019

Try go get -d -u git-codecommit.eu-west-1.amazonaws.com/v1/repos/myRepo.git

See https://golang.org/cmd/go/#hdr-Remote_import_paths

@Baxtex

This comment has been minimized.

Copy link
Author

commented May 21, 2019

Try go get -d -u git-codecommit.eu-west-1.amazonaws.com/v1/repos/myRepo.git

See https://golang.org/cmd/go/#hdr-Remote_import_paths

Adding .git to the end gave me another problem, so at least some progress. :)

package git-codecommit.eu-west-1.amazonaws.com/v1/repos/battery-historian.git: cannot download, git://git-codecommit.eu-west-1.amazonaws.com/v1/repos/myRepo uses insecure protocol

Isn't go get using HTTPS by default? How can that be insercure? If I add either SSH or HTTPS as the protocol, it fails sating the import path is malformed. I've seen solutions on how to specify using SSH for github but not understood how to do that for Code commit. I've tried adding


[url "git@git-codecommit.eu-west-1.amazonaws.com:"]
	insteadOf = https://git-codecommit.eu-west-1.amazonaws.com/v1/repos/

To my .gitconfig file but it did not help.

@Baxtex Baxtex changed the title go get -d -u code commit repository fail go get -d -u on AWS code commit repository fails May 21, 2019

@thepudds

This comment has been minimized.

Copy link

commented May 21, 2019

Perhaps not directly related, but #25792 is older AWS CodeCommit vgo bug, and includes final comment:

It's fixed with one small detail.

Package imports as git-codecommit.us-east-1.amazonaws.com/v1/repos/return2.git require that packages go.mod contains module git-codecommit.us-east-1.amazonaws.com/v1/repos/return2.git.

@Baxtex Probably worth at least skimming that bug.

@bcmills bcmills changed the title go get -d -u on AWS code commit repository fails cmd/go: go get -d -u on AWS code commit repository fails May 21, 2019

@bcmills bcmills added this to the Go1.13 milestone May 21, 2019

@bcmills

This comment has been minimized.

Copy link
Member

commented May 22, 2019

@Baxtex, the go command probes .git paths using all known protocol schemes for the VCS that meet security requirements (see here).

Can you git clone or git ls-remote the https URL for the path?

@Baxtex

This comment has been minimized.

Copy link
Author

commented May 22, 2019

After some more research, I discovered that I probably don't need to do go get/git clone at all. As I'm running this on AWS code build, it already has access to the repository. So I realized I could use the COPY command instead. Here is my updated dockerfile:

#Use latest golang version 
FROM golang:latest 
 
MAINTAINER Me 
 
#Install Java 
RUN apt-get -y update && apt-get install -y openjdk-8-jre-headless 

#Get the battery historian code 
WORKDIR /go/src/codecommit/battery-historian 
COPY . /go/src/codecommit/battery-historian 
 
RUN go get ./... 
#RUN go get -d -u 
RUN go run setup.go 
 
#Open default port 
EXPOSE 8888 
 
#Run the start file for battery historian in a container 
CMD go run cmd/battery-historian/battery-historian.go --port 8888 --name my-battery-historian 

As you can see, I use copy to copy in my own battery historian repository instead of using go get. Do I have to do something more? Because when I study my logs, it looks like it cannot find all my dependencies:


Step 7/9 : RUN go run setup.go 
 ---> Running in f1735a0eb865 
 
Generating JS runfiles... 
Couldn't generate runfile: failed to run command "python /go/src/codecommit/battery-historian/third_party/closure-library/closure/bin/build/depswriter.py --root=/go/src/codecommit/battery-historian/third_party/closure-library/closure/goog --root_with_prefix=js ../../../../js": 
  exit status 2 
  python: can't open file '/go/src/codecommit/battery-historian/third_party/closure-library/closure/bin/build/depswriter.py': [Errno 2] No such file or directory 

For reference, the repo I got on Code commit is a clone of googles battery historian: https://github.com/google/battery-historian

@agnivade

This comment has been minimized.

Copy link
Member

commented Jun 8, 2019

@Baxtex - Have you manually checked that third_party/closure-library/closure/bin/build/depswriter.py exists ?

Could you use rebuild=true and run docker build again ?

And in any case, this does not seem to be a Go issue anymore as you are just copying files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.