git clone does not fetch lfs files

[sophana]

Hi

I'm using git 2.7.1 and git-lfs 1.2.0 under el6 64bit.
I successfully tried git-lfs with gitlab server.
Few days before, I think that a git clone fetched lfs files automatically.
now, I'm not sure what I changed (maybe upgrade git-lfs from 1.1.0?), but lfs files are no more fetched by default.
I have to do "git lfs pull", or clone with "git lfs clone".

Is it normal behavior? Is there something that I need to do for git clone to fetch lfs files?

I'm using http url with credential.helper=store.

I also tried downgrading git-lfs to 1.1.0, but still the same.

[technoweenie]

What does git lfs env show?

[sophana]

$ git lfs env
git-lfs/1.2.0 (GitHub; linux amd64; go 1.5.3; git 9bd3b8e)
git version 2.7.1

Endpoint=http://gitlab/root/ipctest.git/info/lfs (auth=none)
LocalWorkingDir=/home/sophana/src/git2/ipctest1
LocalGitDir=/home/sophana/src/git2/ipctest1/.git
LocalGitStorageDir=/home/sophana/src/git2/ipctest1/.git
LocalMediaDir=/home/sophana/src/git2/ipctest1/.git/lfs/objects
LocalReferenceDir=
TempDir=/home/sophana/src/git2/ipctest1/.git/lfs/tmp
ConcurrentTransfers=3
BatchTransfer=true
git config filter.lfs.smudge = "git-lfs smudge %f"
git config filter.lfs.clean = "git-lfs clean %f"

[technoweenie]

Okay, you don't have the skip smudge setting enabled. Then, Git will only fetch LFS files that match patterns in your .gitattributes.

The reason why git lfs clone works is that git lfs pull doesn't use .gitattributes. If you like that workflow of having to run git lfs pull, look at the git lfs install docs.

https://github.com/github/git-lfs/blob/master/docs/man/git-lfs-install.1.ronn
https://github.com/github/git-lfs/blob/master/docs/man/git-lfs-pull.1.ronn

[technoweenie]

Blah, hit enter too soon. If you have more problems, post a full GIT_TRACE=1 git clone, and the .gitattributes file at the HEAD commit of your clone.

[sophana]

The repo was converted to lfs with BFG
BFG added a .gitattribute file in each directory which contained binary files

*.{raw,dxr,vsd,pdf,docx,xls,doc,xlsx,ppt,pptx,tbz,tgz,tar.gz} filter=lfs diff=lfs merge=lfs -text

I also added one on top with git lfs track *.{raw,dxr,vsd,pdf,docx,xls,doc,xlsx,ppt,pptx,tbz,tgz,tar.gz}

for example pdf files are not fetched.
Maybe the syntax with {} is not understood in .gitattributes?

$ GIT_TRACE=1 git clone http://gitlab/root/ipctest.git ipctest3
17:00:45.049638 git.c:348 trace: built-in: git 'clone' 'http://gitlab/root/ipctest.git' 'ipctest3'
Cloning into 'ipctest3'...
17:00:45.051466 run-command.c:343 trace: run_command: 'git-remote-http' 'origin' 'http://gitlab/root/ipctest.git'
17:00:45.080060 run-command.c:343 trace: run_command: 'fetch-pack' '--stateless-rpc' '--stdin' '--lock-pack' '--thin' '--check-self-contained-and-connected' '--cloning' 'http://gitlab/root/ipctest.git/'
17:00:45.080268 exec_cmd.c:128 trace: exec: 'git' 'fetch-pack' '--stateless-rpc' '--stdin' '--lock-pack' '--thin' '--check-self-contained-and-connected' '--cloning' 'http://gitlab/root/ipctest.git/'
17:00:45.081166 git.c:348 trace: built-in: git 'fetch-pack' '--stateless-rpc' '--stdin' '--lock-pack' '--thin' '--check-self-contained-and-connected' '--cloning' 'http://gitlab/root/ipctest.git/'
remote: Counting objects: 174618, done.
remote: Compressing objects: 100% (35130/35130), done.
17:00:46.130603 run-command.c:343 trace: run_command: 'index-pack' '--stdin' '-v' '--fix-thin' '--keep=fetch-pack 30851 on d5.lcl' '--check-self-contained-and-connected' '--pack_header=2,174618'
17:00:46.130791 exec_cmd.c:128 trace: exec: 'git' 'index-pack' '--stdin' '-v' '--fix-thin' '--keep=fetch-pack 30851 on d5.lcl' '--check-self-contained-and-connected' '--pack_header=2,174618'
17:00:46.131859 git.c:348 trace: built-in: git 'index-pack' '--stdin' '-v' '--fix-thin' '--keep=fetch-pack 30851 on d5.lcl' '--check-self-contained-and-connected' '--pack_header=2,174618'
remote: Total 174618 (delta 137045), reused 174149 (delta 136728)
Receiving objects: 100% (174618/174618), 36.91 MiB | 33.42 MiB/s, done.
Resolving deltas: 100% (137045/137045), done.
Checking connectivity... 17:00:49.294901 run-command.c:343 trace: run_command: 'rev-list' '--objects' '--stdin' '--not' '--all'
17:00:49.295050 exec_cmd.c:128 trace: exec: 'git' 'rev-list' '--objects' '--stdin' '--not' '--all'
17:00:49.296035 git.c:348 trace: built-in: git 'rev-list' '--objects' '--stdin' '--not' '--all'
done.

[technoweenie]

Correct, Git doesn't understand that syntax. See rtyley/bfg-repo-cleaner#116.

I'd recommend trying https://github.com/bozaro/git-lfs-migrate, unless you want to do the painful git filter-branch dance to replace the BFG-created .gitattributes files with the correct one.

[sophana]

I removed all .gitattributes to a single one on top with correct syntax.
It now works fine.

It is really true that git lfs clone is MUCH faster than git clone...

Thanks a lot!

[technoweenie]

@sophana If you only did that on the most recent commit, without another big migration, then any previous commits will not fetch LFS objects. You can always do that by running git lfs pull manually though.

It is really true that git lfs clone is MUCH faster than git clone...

Then you may be interested in the git lfs install --skip-smudge option. You can get the same speed benefits on git pull operations after the initial clone.

https://github.com/github/git-lfs/blob/master/docs/man/git-lfs-install.1.ronn

Glad you god it working :)

[sophana]

I tried git lfs install --skip-smudge and don't see the benefit, because you need to type git lfs pull after the clone.
I'd prefer that people have the files slowly if they forget to do a git lfs clone
We don't manipulate binary files very often, and they are few of them.

I also remade the transfer (from git-svn) with https://github.com/bozaro/git-lfs-migrate. Worked just fine.
Thanks again for the pointer.