New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: go get -u fails with local remote url #13084

Open
dalu opened this Issue Oct 28, 2015 · 12 comments

Comments

Projects
None yet
5 participants
@dalu

dalu commented Oct 28, 2015

I commented in #11457
but this is a new issue, albeit related.

The workflow is this:

  • user is getting created
  • bare repo /home/$user/repo.git is created
  • ~/go/src/project/ is created
  • .ssh/authorized_keys is created
  • code is pushed to server like git remote add s4 user@s4.mydomain.tld:repo:git; git push -u s4 master
  • post-update hook is invoked which clones into ~/go/src/project and does the things mentioned below in the build command (I see that I should update my build command and remove netgo tag)

So there are only 2 possible reasons why this fails:

  1. cmd := exec.Command("go", "get", "-u")
    or
  2. cmd = exec.Command("go", "build", "-o", un, "-a", "-tags", "netgo")
# go version
go version go1.5.1 linux/amd64
# cd /home/gewicht.luketic.de/
s4 gewicht.luketic.de # l
total 40K
drwxr-xr-x 5 gewicht.luketic.de gewicht.luketic.de 4.0K Oct 27 15:07 .
drwxr-xr-x 9 root               root               4.0K Oct 27 02:51 ..
-rw------- 1 gewicht.luketic.de gewicht.luketic.de   78 Oct 27 15:07 .bash_history
-rw-r--r-- 1 gewicht.luketic.de gewicht.luketic.de  127 Aug 19 21:15 .bash_logout
-rw-r--r-- 1 gewicht.luketic.de gewicht.luketic.de  193 Aug 19 21:15 .bash_profile
-rw-r--r-- 1 gewicht.luketic.de gewicht.luketic.de  551 Aug 19 21:15 .bashrc
-rw-r--r-- 1 gewicht.luketic.de gewicht.luketic.de  172 Oct 27 02:51 .controlkey
drwxr-xr-x 5 gewicht.luketic.de gewicht.luketic.de 4.0K Oct 27 15:07 go
drwxr-xr-x 6 gewicht.luketic.de gewicht.luketic.de 4.0K Oct 27 02:51 repo.git
drwx------ 2 gewicht.luketic.de gewicht.luketic.de 4.0K Oct 27 02:51 .ssh
s4 gewicht.luketic.de # tree
.
├── go
│   ├── bin
│   │   └── project
│   ├── pkg
│   │   └── linux_amd64
│   │       ├── github.com
│   │       │   └── gohost
│   │       │       └── paddr.a
│   │       └── gopkg.in
│   │           ├── mgo.v2
│   │           │   ├── bson.a
│   │           │   └── internal
│   │           │       └── scram.a
│   │           └── mgo.v2.a
│   └── src
│       ├── github.com
│       │   └── gohost
│       │       └── paddr
│       │           ├── privateaddr.go
│       │           ├── privateaddr_test.go
│       │           └── README.md
│       ├── gopkg.in
│       │   └── mgo.v2
│       │       ├── auth.go
│       │       ├── auth_test.go
│       │       ├── bson
│       │       │   ├── bson.go
│       │       │   ├── bson_test.go
│       │       │   ├── decode.go
│       │       │   ├── encode.go
│       │       │   ├── LICENSE
│       │       │   ├── specdata
│       │       │   │   └── update.sh
│       │       │   └── specdata_test.go
│       │       ├── bulk.go
│       │       ├── bulk_test.go
│       │       ├── cluster.go
│       │       ├── cluster_test.go
│       │       ├── dbtest
│       │       │   ├── dbserver.go
│       │       │   ├── dbserver_test.go
│       │       │   └── export_test.go
│       │       ├── doc.go
│       │       ├── export_test.go
│       │       ├── gridfs.go
│       │       ├── gridfs_test.go
│       │       ├── internal
│       │       │   ├── sasl
│       │       │   │   ├── sasl.c
│       │       │   │   ├── sasl.go
│       │       │   │   ├── sasl_windows.c
│       │       │   │   ├── sasl_windows.go
│       │       │   │   ├── sasl_windows.h
│       │       │   │   ├── sspi_windows.c
│       │       │   │   └── sspi_windows.h
│       │       │   └── scram
│       │       │       ├── scram.go
│       │       │       └── scram_test.go
│       │       ├── LICENSE
│       │       ├── log.go
│       │       ├── Makefile
│       │       ├── queue.go
│       │       ├── queue_test.go
│       │       ├── raceoff.go
│       │       ├── raceon.go
│       │       ├── README.md
│       │       ├── saslimpl.go
│       │       ├── saslstub.go
│       │       ├── server.go
│       │       ├── session.go
│       │       ├── session_test.go
│       │       ├── socket.go
│       │       ├── stats.go
│       │       ├── suite_test.go
│       │       ├── syscall_test.go
│       │       ├── syscall_windows_test.go
│       │       ├── testdb
│       │       │   ├── client.pem
│       │       │   ├── dropall.js
│       │       │   ├── init.js
│       │       │   ├── server.pem
│       │       │   ├── setup.sh
│       │       │   ├── supervisord.conf
│       │       │   └── wait.js
│       │       ├── testserver
│       │       │   ├── export_test.go
│       │       │   ├── testserver.go
│       │       │   └── testserver_test.go
│       │       └── txn
│       │           ├── chaos.go
│       │           ├── debug.go
│       │           ├── dockey_test.go
│       │           ├── flusher.go
│       │           ├── mgo_test.go
│       │           ├── sim_test.go
│       │           ├── tarjan.go
│       │           ├── tarjan_test.go
│       │           ├── txn.go
│       │           └── txn_test.go
│       └── project
│           ├── gewicht.luketic.de
│           ├── main.go
│           └── static
│               └── index.html
└── repo.git
    ├── config
    ├── description
    ├── HEAD
    ├── hooks
    │   ├── applypatch-msg.sample
    │   ├── commit-msg.sample
    │   ├── post-update
    │   ├── post-update.sample
    │   ├── pre-applypatch.sample
    │   ├── pre-commit.sample
    │   ├── prepare-commit-msg.sample
    │   ├── pre-push.sample
    │   ├── pre-rebase.sample
    │   └── update.sample
    ├── info
    │   ├── exclude
    │   └── refs
    ├── objects
    │   ├── 16
    │   │   └── a9fdee551f95921b142c7f8b52951b8ca08b61
    │   ├── 26
    │   │   └── 2081fd4e3ad664705a0863ce9253850dccb762
    │   ├── 3c
    │   │   └── 7cce1ec59019d444c124b22ba4379a05d594b0
    │   ├── 5b
    │   │   └── 86b2ef84c60c764acc670726b32a4a31bc808c
    │   ├── 72
    │   │   └── 5135ea5625e06588c54a58f81fd7daa91ffe4b
    │   ├── f0
    │   │   └── 23cf1c034a529cde14e1f437195adb2a1f3bd8
    │   ├── info
    │   │   └── packs
    │   └── pack
    └── refs
        ├── heads
        │   └── master
        └── tags

~/go/src/project/.git/config

[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
[remote "origin"]
        url = /home/gewicht.luketic.de/repo.git
        fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
        remote = origin
        merge = refs/heads/master

when building

remote: package project: unable to parse output of git config remote.origin.url
remote: error: exit status 1

the build command:

        case "build":
                if os.Getenv("GOPATH") != os.Getenv("HOME")+"/go" {
                        os.Setenv("GOPATH", os.Getenv("HOME")+"/go")
                }

                if e := os.Chdir(os.Getenv("HOME") + defaultProjectPath + "/"); e != nil {
                        printError(e)
                        return
                }

                cmd := exec.Command("go", "get", "-u")
                cmd.Stdout = os.Stdout
                cmd.Stderr = os.Stderr
                if e := cmd.Start(); e != nil {
                        printError(e)
                        return
                }
                if e := cmd.Wait(); e != nil {
                        printError(e)
                        return
                }
                t := strings.Split(os.Getenv("HOME"), "/")
                un := t[len(t)-1]
                cmd = exec.Command("go", "build", "-o", un, "-a", "-tags", "netgo")
                cmd.Stdout = os.Stdout
                cmd.Stderr = os.Stderr
                if e := cmd.Start(); e != nil {
                        printError(e)
                        return
                }
                if e := cmd.Wait(); e != nil {
                        printError(e)
                        return
                }

@ianlancetaylor ianlancetaylor changed the title from go get -u fails with local remote url to cmd/go: go get -u fails with local remote url Oct 28, 2015

@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Oct 28, 2015

There is a lot of information here but I can't figure out what the actual problem is. Please show exactly how the go tool is being run, exactly what happens, and what you expected to happen instead. Thanks.

@ianlancetaylor ianlancetaylor added this to the Go1.6 milestone Oct 28, 2015

@dalu

This comment has been minimized.

dalu commented Oct 28, 2015

Yes it's a lot to process. Everything is noted in the initial post.

Please show exactly how the go tool is being run

The go tool is being ran by invoking the "build" branch of the switch command.
It's the equivalent of

cd /home/$USER/go/src/project
go get -u
go build -o $USER -a -tags netgo

just in Go, it's what you see as "the build command:"

Basically it's on my personal cluster

I create a user and a bare git repo, which serves as a push target and a source of a git clone into
/home/$USER/go/src/project/

I guess one could reproduce it with:

cd
mkdir local
git init --bare repo.git
mkdir -p ~/go/src/project/
cat <<EOF > local/main.go
package main

import "fmt"

func main() {
fmt.Println("Hello")
}
EOF
cat <<EOF > repo.git/hooks/post-update
#!/bin/bash
git clone /home/username/repo.git /home/username/go/src/project/
cd /home/username/go/src/project/
go get -u
go build -o username -a --tags netgo
EOF
cd local
git init
git add .
git commit -m 'initial'
git remote add deploy /home/username/repo.git
git push -u deploy master

In essence:
the remote.origin.url is /home/username/repo.git
and go get -u for some reason prints this error message
remote: package project: unable to parse output of git config remote.origin.url

so that means that it, I assume the go tool, is unable to
parse the output of the command
git config remote.origin.url
of the package project

So I'm su-ing as a user and cding into ~/go/src/project/

gewicht.luketic.de is the username and see initial post for the git config of ~/go/src/project/ git repo (this is the destination of the clone from repo.git)

gewicht.luketic.de@s4 ~/go/src/project $ go get -u
package _/home/gewicht.luketic.de/go/src/project: unrecognized import path "_/home/gewicht.luketic.de/go/src/project"

this is because the GOPATH isn't set
so,
export GOPATH=/home/gewicht.luketic.de/go/

then

go get

no errors (!)

go get -u
package project: unable to parse output of git config remote.origin.url

so go get -u is giving us an error

go build -o gewicht.luketic.de -a
no errors

just go get -u which fails because the remote is a local bare git repository

@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Oct 28, 2015

What is the output of go get -u -x in the case where it fails?

@dalu

This comment has been minimized.

dalu commented Oct 28, 2015

$ go get -u -x
cd /home/gewicht.luketic.de/go/src/project
git config remote.origin.url
package project: unable to parse output of git config remote.origin.url
@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Oct 28, 2015

Thanks. What is the output of git config remote.origin.url in your repository?

@dalu

This comment has been minimized.

dalu commented Oct 28, 2015

$ git config remote.origin.url
/home/gewicht.luketic.de/repo.git
@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Oct 28, 2015

Thanks, I finally get it. Marking as unplanned since it seems like an unusual case, but would be happy to review a patch.

@ianlancetaylor ianlancetaylor modified the milestones: Unplanned, Go1.6 Oct 28, 2015

@dalu

This comment has been minimized.

dalu commented Oct 28, 2015

Yes I should probably build somewhere else and copy the binary to the cluster.

Does that mean I get to write a patch for go get -u with local bare git repos?
For now my workaround is removing the -u and see if that leads to any errors or not.
It's still quite an experimental platform (my cluster).

@rakyll rakyll self-assigned this Oct 28, 2015

@rakyll

This comment has been minimized.

Member

rakyll commented Oct 29, 2015

I don't think we should ever solve this issue.

How do you go get a local repo in the first place? go get is a tool that is designed to work with remote sources. If you are switching to raw git and add a local remote, you can always fallback to raw git pull to get the updates.

go get -u should not work with local repos if go get is only working with remote sources.

@dalu

This comment has been minimized.

dalu commented Oct 29, 2015

A local repo is a remote source in the sense that its origin is a different directory ;)
Since in git clone source dest the source can be any bare git repo.

The problem is, if I want up to date packages I need to use -u which I understand is short for update, altough it says something like "utilizes network". So it forces a clone or pull or whatever, as long as it's up to date.

How to distinguish? If the url starts with / it's probably a local repo.

Thing is, I don't need an update of the local repo. I don't know why go get -u tries to update itself, but it does and there's this error.
So the issue is, why does it try to update itself in the first place?

@rakyll rakyll removed their assignment Feb 10, 2016

@dolmen

This comment has been minimized.

dolmen commented Mar 27, 2017

Git allows many other string formats in remote.url which are not strictly URLs. For example it also takes unrecognized URL schemes (the part before :) as SSH hostnames and that hostname may be expanded by SSH using information from ~/.ssh/config (I'm using this feature for my tool github-keygen).

I don't think it is possible for go get to support all Git corner cases, but maybe it could just delegate to the git fetch command for the unrecognized URL formats.

@eldondevcg

This comment has been minimized.

eldondevcg commented Aug 11, 2017

I think this is a valid use case. It seems also to not work for the file:// url. The assumption that all filesystem remotes are local may not be correct, as there are ways to expose remote resources as paths within the filesystem. NFS, sshfs, and any distributed block store are a few examples.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment