Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

how to rebase with git lfs? #1287

Closed
greggman opened this issue Jun 3, 2016 · 8 comments
Closed

how to rebase with git lfs? #1287

greggman opened this issue Jun 3, 2016 · 8 comments

Comments

@greggman
Copy link

greggman commented Jun 3, 2016

I've got a repoA with 2 lfs tracked files. I'm bringing that repo into repoB like this

cd repoB
git fetch repoA somebranch
git checkout -b temp FETCH_HEAD
git rebase someotherbranch

note there is no common history between the 2 repos and should be no file conflicts either.
Anyway, this prints several lines of "Applying: ..." and then

Using index info to reconstruct a base tree...
Falling back to patching base and 3-way merge...
error: Your local changes to the following files would be overwritten by merge:
    Assets/HappyFunTimes/HappyFunTimesCore/Server/Resources/HFTOSXServer.bytes
Please, commit your changes or stash them before you can merge.
Aborting
error: Failed to merge in the changes.
Patch failed at 0340 update server
The copy of the patch that failed is found in: .git/rebase-apply/patch

When you have resolved this problem, run "git rebase --continue".
If you prefer to skip this patch, run "git rebase --skip" instead.
To check out the original branch and stop rebasing, run "git rebase --abort".

How do I fix this?

git status showed this

$ git status
rebase in progress; onto 5af1f30
You are currently rebasing branch 'gamepad' on '5af1f30'.
  (all conflicts fixed: run "git rebase --continue")

Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    modified:   Assets/HappyFunTimes/HappyFunTimesCore/Server/Resources/HFTOSXServer.bytes
    deleted:    Assets/HappyFunTimes/HappyFunTimesCore/Server/Resources/HFTOSXServer.sha256.bytes

no changes added to commit (use "git add" and/or "git commit -a")

Note: I tried just adding and committing the 2 files (voodoo) and then git rebase --continue which kept going until the next time the file was modified in the history at which point it got a similar error. I did the same thing and it finally finished. But then when I tried to rebase that onto another branch I got

Downloading Assets/HappyFunTimes/HappyFunTimesCore/Server/Resources/HFTOSXServer.bytes (7.48 MB)
Error downloading object: Assets/HappyFunTimes/HappyFunTimesCore/Server/Resources/HFTOSXServer.bytes (4743b094eeab821140773213ebabdaa81c9ac2eb1be1108e70e8d51ae52873dd)

Errors logged to /Users/gregg/src/hft-unity3d/.git/lfs/objects/logs/20160603T213456.110362284.log
Use `git lfs logs last` to view the log.
error: external filter git-lfs smudge -- %f failed 2
error: external filter git-lfs smudge -- %f failed
fatal: Assets/HappyFunTimes/HappyFunTimesCore/Server/Resources/HFTOSXServer.bytes: smudge filter lfs failed
Could not apply dc378b5d715103e9af0ee805ff2a3be1159739aa... add lfs support

Which kind of suggests I have no clue how to use git lfs correctly.

Is this supposed to work? Did miss a step? Should I have done something in repoB before rebasing? repoB is on the same machine but had no lfs files in it (no .gitattributes). that file doesn't appear until well into the history of repoA (where a large binary file was first needed).

@greggman
Copy link
Author

greggman commented Jun 3, 2016

So .... I tried starting over. I found out git lfs install updates the current repo only? This is not at all clear from the docs on https://git-lfs.github.com/ which say You only have to set up Git LFS once. which suggests git lfs install is system wide not once per repo

so anyway

cd repoB
git lfs install  # -- added
git fetch repoA somebranch
git checkout -b temp FETCH_HEAD
git rebase someotherbranch

still gets the exact same error at the same spot

Using index info to reconstruct a base tree...
Falling back to patching base and 3-way merge...
error: Your local changes to the following files would be overwritten by merge:
    Assets/HappyFunTimes/HappyFunTimesCore/Server/Resources/HFTOSXServer.bytes
Please, commit your changes or stash them before you can merge.
Aborting
error: Failed to merge in the changes.
Patch failed at 0340 update server
The copy of the patch that failed is found in: .git/rebase-apply/patch

When you have resolved this problem, run "git rebase --continue".
If you prefer to skip this patch, run "git rebase --skip" instead.
To check out the original branch and stop rebasing, run "git rebase --abort".

@greggman
Copy link
Author

greggman commented Jun 3, 2016

I also tried doing this

cd repoB
git lfs install  # -- added
git fetch repoA somebranch
git checkout -b temp FETCH_HEAD
git rebase -i someotherbranch  # use interactive mode

Then I set the commit were lfs was first added to edit. That's actually about 30 commits before the failure above. I thought maybe (voodoo again) that I needed to commit .gitattributes before the file it's tracking. As it was there were both committed in the same commit (again it's not at all clear from the docs that I'm even supposed to commit .gitattributes nor is it clear if it should be committed alone or with other files.) In any case I thought I'd split them at that commit by committing gitattributes first.

Unfortunately rebase failed at the edit (like I mentioned about 30 commits before it was getting the error previously

Downloading Assets/HappyFunTimes/HappyFunTimesCore/Server/Resources/HFTOSXServer.bytes (7.48 MB)
Error downloading object: Assets/HappyFunTimes/HappyFunTimesCore/Server/Resources/HFTOSXServer.bytes (4743b094eeab821140773213ebabdaa81c9ac2eb1be1108e70e8d51ae52873dd)

Errors logged to /Users/gregg/src/hft-unity3d/.git/lfs/objects/logs/20160603T224422.377101299.log
Use `git lfs logs last` to view the log.
error: external filter git-lfs smudge -- %f failed 2
error: external filter git-lfs smudge -- %f failed
fatal: Assets/HappyFunTimes/HappyFunTimesCore/Server/Resources/HFTOSXServer.bytes: smudge filter lfs failed
Could not apply 657819359e8d3aabe807f24dcf438d71b2f9b2cd... add lfs support

I see the add lfs support but you can see above I already added it.

The log says

$ git lfs logs last
git-lfs/1.2.0 (GitHub; darwin amd64; go 1.6.1; git 9bd3b8e)
git version 2.7.4 (Apple Git-66)

$ git-lfs smudge -- Assets/HappyFunTimes/HappyFunTimesCore/Server/Resources/HFTOSXServer.bytes
Error downloading object: Assets/HappyFunTimes/HappyFunTimesCore/Server/Resources/HFTOSXServer.bytes (4743b094eeab821140773213ebabdaa81c9ac2eb1be1108e70e8d51ae52873dd)

Object not found on the server.
goroutine 1 [running]:
github.com/github/git-lfs/lfs.Stack(0x0, 0x0, 0x0)
    /Users/rick/go/src/github.com/github/git-lfs/lfs/errors.go:566 +0x80
github.com/github/git-lfs/commands.logPanicToWriter(0x98d330, 0xc820180010, 0x990f60, 0xc82044a2e0)
    /Users/rick/go/src/github.com/github/git-lfs/commands/commands.go:195 +0xf80
github.com/github/git-lfs/commands.logPanic(0x990f60, 0xc82044a2e0, 0x0, 0x0)
    /Users/rick/go/src/github.com/github/git-lfs/commands/commands.go:159 +0x412
github.com/github/git-lfs/commands.handlePanic(0x990f60, 0xc82044a2e0, 0x0, 0x0)
    /Users/rick/go/src/github.com/github/git-lfs/commands/commands.go:134 +0x4e
github.com/github/git-lfs/commands.LoggedError(0x990f60, 0xc82044a2e0, 0x617540, 0x21, 0xc820163c98, 0x2, 0x2)
    /Users/rick/go/src/github.com/github/git-lfs/commands/commands.go:84 +0x82
github.com/github/git-lfs/commands.smudgeCommand(0x8467a0, 0xc8200bf500, 0x1, 0x2)
    /Users/rick/go/src/github.com/github/git-lfs/commands/command_smudge.go:79 +0xdcb
github.com/github/git-lfs/vendor/_nuts/github.com/spf13/cobra.(*Command).execute(0x8467a0, 0xc8200bf360, 0x2, 0x2, 0x0, 0x0)
    /Users/rick/go/src/github.com/github/git-lfs/vendor/_nuts/github.com/spf13/cobra/command.go:477 +0x3f4
github.com/github/git-lfs/vendor/_nuts/github.com/spf13/cobra.(*Command).Execute(0x8477e0, 0x0, 0x0)
    /Users/rick/go/src/github.com/github/git-lfs/vendor/_nuts/github.com/spf13/cobra/command.go:551 +0x46a
github.com/github/git-lfs/commands.Run()
    /Users/rick/go/src/github.com/github/git-lfs/commands/commands.go:99 +0x23
main.main()
    /Users/rick/go/src/github.com/github/git-lfs/git-lfs.go:34 +0x12e

ENV:
LocalWorkingDir=/Users/gregg/src/hft-unity3d
LocalGitDir=/Users/gregg/src/hft-unity3d/.git
LocalGitStorageDir=/Users/gregg/src/hft-unity3d/.git
LocalMediaDir=/Users/gregg/src/hft-unity3d/.git/lfs/objects
LocalReferenceDir=
TempDir=/Users/gregg/src/hft-unity3d/.git/lfs/tmp
ConcurrentTransfers=3
BatchTransfer=true
GIT_INTERNAL_GETTEXT_SH_SCHEME=fallthrough
GIT_DIR=.git
GIT_CHERRY_PICK_HELP=
When you have resolved this problem, run "git rebase --continue".
If you prefer to skip this patch, run "git rebase --skip" instead.
To check out the original branch and stop rebasing, run "git rebase --abort".

GIT_REFLOG_ACTION=rebase -i (edit)
GIT_PREFIX=

@greggman
Copy link
Author

greggman commented Jun 3, 2016

latest attempt I started over.

I clone repoA from githib

git clone git@github.com:me/repoa.git
cd repoA

then I added lfs

git lfs install

I then added repoB as a remote

git remote add repoB git@github.com:me/repoB.git

I then fetched all of repoB

git fetch reboB

I then made a branch based on repoB

git checkout -b temp repoB/somebranch

that seems closer to what I'm supposed to do but it also failed

$ git checkout -b temp repoB/somebranch
Downloading Assets/HappyFunTimes/HappyFunTimesCore/Server/Resources/HFTOSXServer.bytes (7.48 MB)
Error downloading object: Assets/HappyFunTimes/HappyFunTimesCore/Server/Resources/HFTOSXServer.bytes (f8c42a7c55f610768ce50ff93d09fc63fa897de867290dafee2e84d64e10de4e)

Errors logged to /Users/gregg/temp/delme-hft-unity3d/.git/lfs/objects/logs/20160603T231351.670335751.log
Use `git lfs logs last` to view the log.
error: external filter git-lfs smudge -- %f failed 2
error: external filter git-lfs smudge -- %f failed
fatal: Assets/HappyFunTimes/HappyFunTimesCore/Server/Resources/HFTOSXServer.bytes: smudge filter lfs failed

it's the same error as before

$ git lfs logs last
git-lfs/1.2.0 (GitHub; darwin amd64; go 1.6.1; git 9bd3b8e)
git version 2.7.4 (Apple Git-66)

$ git-lfs smudge -- Assets/HappyFunTimes/HappyFunTimesCore/Server/Resources/HFTOSXServer.bytes
Error downloading object: Assets/HappyFunTimes/HappyFunTimesCore/Server/Resources/HFTOSXServer.bytes (f8c42a7c55f610768ce50ff93d09fc63fa897de867290dafee2e84d64e10de4e)

Object not found on the server.
goroutine 1 [running]:
github.com/github/git-lfs/lfs.Stack(0x0, 0x0, 0x0)
    /Users/rick/go/src/github.com/github/git-lfs/lfs/errors.go:566 +0x80
github.com/github/git-lfs/commands.logPanicToWriter(0x98d330, 0xc82017c010, 0xf14218, 0xc8203dd0a0)
    /Users/rick/go/src/github.com/github/git-lfs/commands/commands.go:195 +0xf80
github.com/github/git-lfs/commands.logPanic(0xf14218, 0xc8203dd0a0, 0x0, 0x0)
    /Users/rick/go/src/github.com/github/git-lfs/commands/commands.go:159 +0x412
github.com/github/git-lfs/commands.handlePanic(0xf14218, 0xc8203dd0a0, 0x0, 0x0)
    /Users/rick/go/src/github.com/github/git-lfs/commands/commands.go:134 +0x4e
github.com/github/git-lfs/commands.LoggedError(0xf14218, 0xc8203dd0a0, 0x617540, 0x21, 0xc82015dc98, 0x2, 0x2)
    /Users/rick/go/src/github.com/github/git-lfs/commands/commands.go:84 +0x82
github.com/github/git-lfs/commands.smudgeCommand(0x8467a0, 0xc8200bd480, 0x1, 0x2)
    /Users/rick/go/src/github.com/github/git-lfs/commands/command_smudge.go:79 +0xdcb
github.com/github/git-lfs/vendor/_nuts/github.com/spf13/cobra.(*Command).execute(0x8467a0, 0xc8200bd2e0, 0x2, 0x2, 0x0, 0x0)
    /Users/rick/go/src/github.com/github/git-lfs/vendor/_nuts/github.com/spf13/cobra/command.go:477 +0x3f4
github.com/github/git-lfs/vendor/_nuts/github.com/spf13/cobra.(*Command).Execute(0x8477e0, 0x0, 0x0)
    /Users/rick/go/src/github.com/github/git-lfs/vendor/_nuts/github.com/spf13/cobra/command.go:551 +0x46a
github.com/github/git-lfs/commands.Run()
    /Users/rick/go/src/github.com/github/git-lfs/commands/commands.go:99 +0x23
main.main()
    /Users/rick/go/src/github.com/github/git-lfs/git-lfs.go:34 +0x12e

ENV:
LocalWorkingDir=/Users/gregg/temp/delme-hft-unity3d
LocalGitDir=/Users/gregg/temp/delme-hft-unity3d/.git
LocalGitStorageDir=/Users/gregg/temp/delme-hft-unity3d/.git
LocalMediaDir=/Users/gregg/temp/delme-hft-unity3d/.git/lfs/objects
LocalReferenceDir=
TempDir=/Users/gregg/temp/delme-hft-unity3d/.git/lfs/tmp
ConcurrentTransfers=3
BatchTransfer=true
GIT_DIR=.git
GIT_PREFIX=

@technoweenie
Copy link
Contributor

So .... I tried starting over. I found out git lfs install updates the current repo only? This is not at all clear from the docs on https://git-lfs.github.com/ which say You only have to set up Git LFS once.

git lfs install adds some internal git filters to your global git config (which is stored in your user directory, ex: ~/.gitconfig). LFS does rely on some git hooks in the .git/hooks/* directory local to your repository. Git doesn't provide a way to automatically set it up, so a number of LFS commands do this step for you, including install.

I think the bigger thing here is that the LFS docs are woefully inadequate, and some kind of User Guide is needed. That is definitely on the roadmap.

Regarding your error at the end:

Error downloading object: path/to/file (f8c42a7c55f610768ce50ff93d09fc63fa897de867290dafee2e84d64e10de4e)
Object not found on the server.

That simply means your LFS remote doesn't have that f8c42a7c55f610768ce50ff93d09fc63fa897de867290dafee2e84d64e10de4e object. You can push this from the local repository that has it (it'd be in .git/lfs/objects/f8/c4) with git lfs push --all. This should happen through a .git/hooks/pre-push hook, which gets installed by various LFS commands as I mentioned above. Based on your git lfs install question, I'm assuming that hook didn't exist, and you were able to push to your git server without uploading the LFS objects.

@greggman
Copy link
Author

greggman commented Jun 3, 2016

Thanks. But I'm still kind of lost. git lfs push --all from? repoA? repoA's lfs files are pushed AFAIK

cd repoA
git lfs push --all origin no-node-needed
Git LFS: (0 of 3 files, 3 skipped) 0 B / 22.45 MB, 22.45 MB skipped  

From repoB? I get an error.

I tried started over like this

 git clone git@github.com:me/repoA.git

This works. I see it get the LFS files so it must be new repos just work but. Ok, let's merge A into B

 git clone git@github.com:me/repoB.git
 git remote add repoA git@github.com:me/repoA.git
 git fetch repoA
 git checkout -b temp repoA/somebranch

fails

Okay try again

 git clone git@github.com:me/repoB.git
 git remote remove origin
 git remote add origin git@github.com:me/repoA.git
 git fetch origin
 git checkout -b temp origin/somebranch

This works. I see lfs download the file. Apparently lfs only works with origin?

Okay let's rebase

 git rebase master

Fails with the same errors as above

Ok maybe because master is tracking repoB?

git branch --set-upstream-to origin/somebranch temp
git rebase master

same error as above

Ideas?

@technoweenie
Copy link
Contributor

Unfortunately, there's no way to pass the git remote down from a git pull {remote} {ref} command to the smudge filter that downloads LFS objects. Your best option is to run install with the --skip-smudge option. This means you'll have to run git lfs pull to download the objects. It's an extra command, but it downloads files more efficiently, and accepts a remote parameter.

@greggman
Copy link
Author

greggman commented Jun 3, 2016

Thanks! That seemd to do it. So just for the next person this seemed to work

 git lfs install --skip-smudge
 git clone git@github.com:me/repoB.git   # repo with no lfs files
 git lfs install --skip-smudge # just to be safe
 git remote add repoA git@github.com:me/repoA.git
 git fetch repoA
 git checkout -b temp repoA/somebranch
 git rebase master
 git lfs pull repoA
 git lfs fetch --all repoA
 git push origin temp
 git lfs push --all origin temp
 git lfs install --force # hoping this removes the --skip-smudge

@greggman greggman closed this as completed Jun 3, 2016
@technoweenie
Copy link
Contributor

This may work a little better:

# This disables smudging for the 'git clone'
# and then calls 'git lfs pull' for you
git lfs clone git@github.com:me/repoB.git
cd repoB
git lfs install --skip-smudge --local # affects only this clone

git fetch repoA
git checkout -b temp repoA/somebranch
git rebase master

git lfs fetch --all repoA
git lfs checkout
git push origin temp

git lfs push --all origin temp
git lfs install --force --local

The git lfs pull command is essentially the same as calling git lfs fetch (downloads LFS objects) and git lfs checkout (copies the locally downloaded files to your working directory). So, my example only downloads objects through the git lfs fetch --all command.

If you want to disable the smudge filter for a single command, you can also use GIT_LFS_SKIP_SMUDGE:

$ GIT_LFS_SKIP_SMUDGE=1 git pull
$ git lfs pull

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants