Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Got error when trying to push large files. #774

Closed
mbcraft opened this issue Oct 23, 2015 · 9 comments
Closed

Got error when trying to push large files. #774

mbcraft opened this issue Oct 23, 2015 · 9 comments

Comments

@mbcraft
Copy link

mbcraft commented Oct 23, 2015

I initialized the repository and added the tracked extensions as explained by the docs. I added the files using the git command "git add images/", commited them with "git commit -m 'my comment bla bla bla'" and then tryed to push. I can't use "git lfs push origin master", but only "git-lfs push origin master".
This is the error i got :

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x10 pc=0x47b605]

goroutine 29 [running]:
github.com/github/git-lfs/lfs.doApiBatchRequest(0xc820514000, 0x13, 0x0, 0x0, 0x0, 0x0, 0x0)
    /Users/rick/go/src/github.com/github/git-lfs/lfs/client.go:399 +0x95
github.com/github/git-lfs/lfs.Batch(0xc820112000, 0x64, 0x64, 0x889e30, 0x6, 0x0, 0x0, 0x0, 0x0, 0x0)
    /Users/rick/go/src/github.com/github/git-lfs/lfs/client.go:199 +0x639
github.com/github/git-lfs/lfs.(*TransferQueue).batchApiRoutine(0xc820354000)
    /Users/rick/go/src/github.com/github/git-lfs/lfs/transfer_queue.go:205 +0x436
created by github.com/github/git-lfs/lfs.(*TransferQueue).run
    /Users/rick/go/src/github.com/github/git-lfs/lfs/transfer_queue.go:328 +0x2b7

goroutine 1 [chan send]:
github.com/github/git-lfs/lfs.(*Batcher).Add(0xc82000f660, 0x7faa69326510, 0xc8203a4f80)
    /Users/rick/go/src/github.com/github/git-lfs/lfs/batcher.go:38 +0xeb
github.com/github/git-lfs/lfs.(*TransferQueue).Add(0xc820354000, 0x7faa69326510, 0xc8203a4f80)
    /Users/rick/go/src/github.com/github/git-lfs/lfs/transfer_queue.go:71 +0xe0
github.com/github/git-lfs/commands.uploadPointers(0xc8200fa000, 0xa85, 0xa85, 0xc8200fa000)
    /Users/rick/go/src/github.com/github/git-lfs/commands/command_push.go:106 +0x725
github.com/github/git-lfs/commands.uploadsBetweenRefAndRemote(0x7ffe3fb8fa14, 0x6, 0xc82000ef50, 0x1, 0x1, 0x755540)
    /Users/rick/go/src/github.com/github/git-lfs/commands/command_push.go:73 +0x7be
github.com/github/git-lfs/commands.pushCommand(0xaaec60, 0xc82000ef40, 0x2, 0x2)
    /Users/rick/go/src/github.com/github/git-lfs/commands/command_push.go:197 +0x657
github.com/github/git-lfs/vendor/_nuts/github.com/spf13/cobra.(*Command).execute(0xaaec60, 0xc82000ed60, 0x2, 0x2, 0x0, 0x0)
    /Users/rick/go/src/github.com/github/git-lfs/vendor/_nuts/github.com/spf13/cobra/command.go:477 +0x403
github.com/github/git-lfs/vendor/_nuts/github.com/spf13/cobra.(*Command).Execute(0xaafb00, 0x0, 0x0)
    /Users/rick/go/src/github.com/github/git-lfs/vendor/_nuts/github.com/spf13/cobra/command.go:551 +0x46a
github.com/github/git-lfs/commands.Run()
    /Users/rick/go/src/github.com/github/git-lfs/commands/commands.go:88 +0x23
main.main()
    /Users/rick/go/src/github.com/github/git-lfs/git-lfs.go:9 +0x14

goroutine 23 [chan receive]:
github.com/github/git-lfs/lfs.(*TransferQueue).errorCollector(0xc820354000)
    /Users/rick/go/src/github.com/github/git-lfs/lfs/transfer_queue.go:256 +0x5b
created by github.com/github/git-lfs/lfs.(*TransferQueue).run
    /Users/rick/go/src/github.com/github/git-lfs/lfs/transfer_queue.go:317 +0x3f

goroutine 24 [chan receive]:
github.com/github/git-lfs/lfs.(*TransferQueue).retryCollector(0xc820354000)
    /Users/rick/go/src/github.com/github/git-lfs/lfs/transfer_queue.go:263 +0x58
created by github.com/github/git-lfs/lfs.(*TransferQueue).run
    /Users/rick/go/src/github.com/github/git-lfs/lfs/transfer_queue.go:318 +0x64

goroutine 25 [chan receive]:
github.com/github/git-lfs/lfs.(*TransferQueue).transferWorker(0xc820354000)
    /Users/rick/go/src/github.com/github/git-lfs/lfs/transfer_queue.go:270 +0x98
created by github.com/github/git-lfs/lfs.(*TransferQueue).run
    /Users/rick/go/src/github.com/github/git-lfs/lfs/transfer_queue.go:322 +0x167

goroutine 26 [chan receive]:
github.com/github/git-lfs/lfs.(*TransferQueue).transferWorker(0xc820354000)
    /Users/rick/go/src/github.com/github/git-lfs/lfs/transfer_queue.go:270 +0x98
created by github.com/github/git-lfs/lfs.(*TransferQueue).run
    /Users/rick/go/src/github.com/github/git-lfs/lfs/transfer_queue.go:322 +0x167

goroutine 27 [chan receive]:
github.com/github/git-lfs/lfs.(*TransferQueue).transferWorker(0xc820354000)
    /Users/rick/go/src/github.com/github/git-lfs/lfs/transfer_queue.go:270 +0x98
created by github.com/github/git-lfs/lfs.(*TransferQueue).run
    /Users/rick/go/src/github.com/github/git-lfs/lfs/transfer_queue.go:322 +0x167

goroutine 28 [chan send]:
github.com/github/git-lfs/lfs.(*Batcher).acceptInput(0xc82000f660)
    /Users/rick/go/src/github.com/github/git-lfs/lfs/batcher.go:72 +0x12e
created by github.com/github/git-lfs/lfs.NewBatcher
    /Users/rick/go/src/github.com/github/git-lfs/lfs/batcher.go:26 +0xdd
@mbcraft
Copy link
Author

mbcraft commented Oct 23, 2015

Also, running "git lfs smudge" returns this error :

Cannot read from STDIN. This command should be run by the Git 'smudge' filter

@mbcraft
Copy link
Author

mbcraft commented Oct 23, 2015

"git lfs pull" returns the following :

(0 of 2693 files) 0 B / 11.22 GB


Anyway, i'm updating the system to the latest version, i'll retry when everything is upgraded.

@mbcraft
Copy link
Author

mbcraft commented Oct 23, 2015

After upgrade nothing has changed.

"git lfs ls-files" correctly outputs the file names :

....
46f3e22653 * images/DSCN2240.JPG
24a9beea73 * images/DSCN2241.JPG
7e2b3055de * images/DSCN2242.JPG
....


"git lfs status" outputs the following :

On branch master
Git LFS objects to be pushed to origin/master:

    images/DSCN0001.JPG (3.9 MB)
    images/DSCN0002.JPG (4.0 MB)
    images/DSCN0003.JPG (4.5 MB)

....

Git LFS objects to be committed:

Git LFS objects not staged for commit:


@rubyist
Copy link
Contributor

rubyist commented Oct 23, 2015

Hi, can you give us the output of git lfs env from your repo directory?

@technoweenie
Copy link
Contributor

I can't use "git lfs push origin master"

Git automatically proxies git {subcmd} to git-{subcmd} if it can find it in the system path. This is unrelated to the panic though. We just need the git-lfs version so we can trace the panic and fix it. Does git-lfs push origin master always panic when you run it from this repository?

@mbcraft
Copy link
Author

mbcraft commented Oct 23, 2015

Version is 1.0.0 .

My files strangely appears on the repo but git push has not uploaded the
real data.

The repo is here https://github.com/mbcraft/trash-data

Kind regards.

-Marco B.

[---------------------------------------
[MBCRAFT di Marco Bagnaresi - Software development services
[Via ca' del vento, 11
[CAP 48012 - Bagnacavallo(RA)
[P.IVA/VAT N. : 02475790396
[C.F. : BGNMRC82H03E730O
[Website : http://www.mbcraft.it
[Standard mail : info@mbcraft.it
[PEC : mbcraft@pec.it
[---------------------------------------
[
[
[Informativa ai sensi del codice della privacy (D.Lgs 196-30/06/2003)
[Questo messaggio è destinato unicamente ai soggetti indicati
nell'intestazione.
[In caso di ricezione erronea si prega di informare il mittente e di eliminare
[il messaggio secondo le norme relative al trattamento dei dati personali.
[La diffusione impropria in qualsiasi forma delle informazioni contenute in
questo messaggio sono proibite
[dalla legge. Questo messaggio può contenere informazioni non corrette.
[Può rivolgersi alla nostra azienda per richiedere informazioni su titolare
e responsabili del trattamento e
[per esercitare i diritti previsti dall'Art.7 del D.lg.196/2003
[inviando una mail con oggetto “Privacy”.
[

Il 23 ottobre 2015 13:59:33 risk danger olson notifications@github.com ha
scritto:

I can't use "git lfs push origin master"

Git automatically proxies git {subcmd} to git-{subcmd} if it can find
it in the system path. This is unrelated to the panic though. We just need
the git-lfs version so we can trace the panic and fix it. Does git-lfs push origin master always panic when you run it from this repository?


Reply to this email directly or view it on GitHub:
#774 (comment)

@technoweenie
Copy link
Contributor

Can you include the full version string?

$ git lfs env
git-lfs/1.0.0 (GitHub; darwin amd64; go 1.5.1; git 4cd71f6)
git version 2.4.9 (Apple Git-60)

Does git-lfs push origin master always panic when you run it from this repository?

@technoweenie
Copy link
Contributor

Ok assuming you're on the v1.0.0 tag, the errors are coming from:

github.com/github/git-lfs/lfs.doApiBatchRequest(0xc820514000, 0x13, 0x0, 0x0, 0x0, 0x0, 0x0)
/Users/rick/go/src/github.com/github/git-lfs/lfs/client.go:399 +0x95
github.com/github/git-lfs/lfs.Batch(0xc820112000, 0x64, 0x64, 0x889e30, 0x6, 0x0, 0x0, 0x0, 0x0, 0x0)
/Users/rick/go/src/github.com/github/git-lfs/lfs/client.go:199 +0x639

Specifically:

func doApiBatchRequest(req *http.Request) (*http.Response, []*objectResource, error) {
    res, err := doAPIRequest(req, Config.PrivateAccess())

    if err != nil {
        if res.StatusCode == 401 {
            return res, nil, newAuthError(err)
        }
        return res, nil, err
    }

res inside of the err != nil block is nil. We're actually breaking standard go conventions here, by assuming that doAPIRequest() always returns a res AND an err, because of #488. A fix for this specific bug is easy: #775. For v1.1, we should do a more thorough review and stop going against go conventions.

I'm also really curious why res is nil. That points to some underlying issue before Git LFS can even receive a response from the LFS API. Would you be interested in trying a custom build to see if it works? I just need to know your platform if you don't want to build it yourself. My hunch is that it will expose an underlying issue that's causing this panic.

@mbcraft
Copy link
Author

mbcraft commented Oct 23, 2015

I think i found where the problem was : git version is 1.7.9.5 . I thought that the full "apt-get update && apt-get upgrade" pushed git version to an enough recent one, but it wasn't true. Now i'm doing a dist upgrade from a LinuxMint/maya to the next one.

Anyway this is the full output :

git-lfs/1.0.0 (GitHub; linux amd64; go 1.5.1; git 7379fa9)
git version 1.7.9.5

LocalWorkingDir=
LocalGitDir=
LocalGitStorageDir=
LocalMediaDir=
TempDir=/tmp/git-lfs
ConcurrentTransfers=3
BatchTransfer=true
git config filter.lfs.smudge = "git-lfs smudge %f"
git config filter.lfs.clean = "git-lfs clean %f"

@mbcraft mbcraft closed this as completed Oct 23, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants