reference not found errors when using bitbucket server

[AlessandroEmm]

Hey there

First and foremost thank you for the library.

I've been trying to clone repos from my private bitbucket server and I constantly run into
"reference not found" errors. I was able to find out that its due to go-git requiring a HEAD reference which bitbucket server doesn't provide by default. Can you explain why HEAD is expected? Ignoring HEAD allows cloning of the particular repo.

Thanks a lot
-Alessandro

[mcuadros]

@AlessandroEmm are you using V3? or V4?

[AlessandroEmm]

Sorry V4.

[mcuadros]

Can you provide the code? HEAD is not required, was a requirement on the old versions, but now in v4.

[AlessandroEmm]

func initRepo(interval int) *git.Repository {
    r := git.NewMemoryRepository()
    o := &git.CloneOptions{
    URL: os.Getenv("REPOSITORYURL"),
    ReferenceName: "refs/heads/master",
    RemoteName: "origin",
    SingleBranch: true,
    }
    if err := r.Clone(o); err != nil {
        time.Sleep(time.Duration(interval) * time.Second)
        fmt.Fprintln(os.Stderr, "Couldn't Clone, retrying in " + strconv.FormatInt(int64(interval), 10) +" seconds...", err)
        return initRepo(interval * 2)
    }

That's how I use it.

And here's how I "fixed" it in your code:

func (r ReferenceStorage) Reference(n plumbing.ReferenceName) (*plumbing.Reference, error) {
    fmt.Println(r, n)
    ref, ok := r[n]
    if !ok && n != "HEAD" {
        return nil, plumbing.ErrReferenceNotFound
    }

    return ref, nil
}

[mcuadros]

Sorry I don't understand the problem, I just executed the the basic example with a BitBucket repository and works perfect

[AlessandroEmm]

I wasn't referring to a public bitbucket repository, but a on-premise bitbucket server formerly known as stash.
But whatever the service when I chose to clone a public github or bitbucket repository:

I get these reference when i fetch from those:
map[refs/heads/master:3e92f0502e373f35e9b6066bfb121b3e5356bec8 refs/heads/master HEAD:ref: refs/heads/master HEAD]

When I fetch from my private git repo on the Bitbucket server i get similar reference minus the HEAD. So my question is why is a reference HEAD required even thought i specify the master branch as the one to check out?

[mcuadros]

I need to replicate it, the HEAD on a remote should not a requirement, in local is created when the clone is done.

[AlessandroEmm]

I just found out with the

fmt.Println(r, n)

from the code above which showed me that the function "Reference" on the ReferenceStorage struct gets called with HEAD as parameter. which then lead to the plumbing.ErrReferenceNotFound.

Hope that helps.

[mcuadros]

@AlessandroEmm I need the following things:

• A small small script replicating the issue
• The full error when you are executing the small script with your private repository

I really don't understand why you say that HEAD is required, the HEAD is not something required is something created by Repository.Clone, since you are providing a branch, your HEAD reference should point to a local branch, with the same name

[AlessandroEmm]

@mcuadros

Can you elaborate what is yet unclear? I will gladly provide you with a script, but I do wonder what I can point out in a script that i haven't posted to this conversation before.

So what I can point out in addition is the actual output of the print statement from above when being run against a public repo vs my private repo:

public
(https://bitbucket.org/Eintrag/metodolog-a-de-la-programaci-n.git)
map[refs/heads/master:3e92f0502e373f35e9b6066bfb121b3e5356bec8 refs/heads/master HEAD:ref: refs/heads/master HEAD] HEAD
map[refs/heads/master:3e92f0502e373f35e9b6066bfb121b3e5356bec8 refs/heads/master HEAD:ref: refs/heads/master HEAD] refs/heads/master

private
map[refs/heads/master:742619f0eaa10fc23f9d88b034530291c12e1037 refs/heads/master] refs/heads/master
map[refs/heads/master:742619f0eaa10fc23f9d88b034530291c12e1037 refs/heads/master] HEAD

As you can see in both cases the cloning via client always assumes a HEAD reference. Which causes problems with my private repo because it doesn't have a reference for HEAD by default (which the public repos seem to have).

[mcuadros]

@AlessandroEmm

For example in your last message, I don't see any error. The HEAD is not a requirement, if something is asking for it, and having an unexpected error please create a script duplicating this error.

Your are cloning a repository to another, so the local repository is having a HEAD because is created by the Clone method.

If you are cloning using SingleBranch you should get this local refs:

742619f0eaa10fc23f9d88b034530291c12e1037 HEAD
742619f0eaa10fc23f9d88b034530291c12e1037 refs/heads/master
742619f0eaa10fc23f9d88b034530291c12e1037 refs/remotes/origin/master

As is tested on this lines https://github.com/src-d/go-git/blob/master/repository_test.go#L119

I really want to help you but, I don't see how

[AlessandroEmm]

You don't see any error because i modified the If statement in your code as per the code above:

if !ok && n != "HEAD" { return nil, plumbing.ErrReferenceNotFound }

The HEAD exclusion prevents me from running into the "reference not found" error.

[mcuadros]

@AlessandroEmm I really need the error, and the code, I cannot make nothing without it

[AlessandroEmm]

@mcuadros

My error is "reference not found"?

[mcuadros]

Hi @AlessandroEmm, I was testing with a private repository at Bitbucket, and I got this error:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x4b1193]

goroutine 1 [running]:
panic(0x6b91e0, 0xc42000c100)
        /usr/lib/go/src/runtime/panic.go:500 +0x1a1
gopkg.in/src-d/go-git.v4/plumbing/format/packfile.(*Decoder).recallByHash(0xc420163cb8, 0x23b637a9fcd4b1bf, 0xce66db876bf06abe, 0x90bc9a2, 0xc44320e20e, 0x0, 0x0, 0xc4203d2bc0)
        /home/mcuadros/workspace/go/src/gopkg.in/src-d/go-git.v4/plumbing/format/packfile/decoder.go:279 +0x43
gopkg.in/src-d/go-git.v4/plumbing/format/packfile.(*Decoder).fillREFDeltaObjectContent(0xc420163cb8, 0x863000, 0xc4203d2bc0, 0x23b637a9fcd4b1bf, 0xce66db876bf06abe, 0x90bc9a2, 0x10, 0x10, 0xc42040c4c0)
        /home/mcuadros/workspace/go/src/gopkg.in/src-d/go-git.v4/plumbing/format/packfile/decoder.go:226 +0xe6
gopkg.in/src-d/go-git.v4/plumbing/format/packfile.(*Decoder).ReadObject(0xc420163cb8, 0x863000, 0xc4203d2b40, 0x23b637a9fcd4b1bf, 0xce66db876bf06abe)
        /home/mcuadros/workspace/go/src/gopkg.in/src-d/go-git.v4/plumbing/format/packfile/decoder.go:162 +0x465
gopkg.in/src-d/go-git.v4/plumbing/format/packfile.(*Decoder).readObjectsWithObjectStorerTx(0xc420163cb8, 0x28, 0xc4200159c0, 0x0)
        /home/mcuadros/workspace/go/src/gopkg.in/src-d/go-git.v4/plumbing/format/packfile/decoder.go:127 +0xc6
gopkg.in/src-d/go-git.v4/plumbing/format/packfile.(*Decoder).doDecode(0xc420163cb8, 0x6a2180, 0x7fc2275709d0)
        /home/mcuadros/workspace/go/src/gopkg.in/src-d/go-git.v4/plumbing/format/packfile/decoder.go:92 +0xa8
gopkg.in/src-d/go-git.v4/plumbing/format/packfile.(*Decoder).Decode(0xc420163cb8, 0x0, 0x0, 0x0, 0xc4203ec630, 0x1000)
        /home/mcuadros/workspace/go/src/gopkg.in/src-d/go-git.v4/plumbing/format/packfile/decoder.go:74 +0x4a
gopkg.in/src-d/go-git%2ev4.(*Remote).updateObjectStorage(0xc42011a330, 0x85cca0, 0xc4204187c0, 0x0, 0x0)
        /home/mcuadros/workspace/go/src/gopkg.in/src-d/go-git.v4/remote.go:198 +0x646
gopkg.in/src-d/go-git%2ev4.(*Remote).Fetch(0xc42011a330, 0xc420163e30, 0x0, 0x0)
        /home/mcuadros/workspace/go/src/gopkg.in/src-d/go-git.v4/remote.go:110 +0x2d8
gopkg.in/src-d/go-git%2ev4.(*Repository).Clone(0xc420163ec0, 0xc420163ed8, 0x0, 0x0)
        /home/mcuadros/workspace/go/src/gopkg.in/src-d/go-git.v4/repository.go:159 +0x29f
main.main()
        /home/mcuadros/workspace/go/src/gopkg.in/src-d/go-git.v4/examples/custom/main.go:16 +0x312

Using this code:

package main

import (
	"os"

	git "gopkg.in/src-d/go-git.v4"
)

func main() {
	r := git.NewMemoryRepository()
	o := &git.CloneOptions{
		URL:           os.Getenv("REPOSITORYURL"),
		ReferenceName: "refs/heads/master",
		SingleBranch:  true,
	}
	if err := r.Clone(o); err != nil {
		panic(err)
	}
}

This is fix on the following PR, can you try if this work with your repo as well?
#130