Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The Git init & fetch approach breaks Git LFS #640

Closed
strich opened this issue Sep 5, 2015 · 28 comments

Comments

@strich
Copy link
Contributor

commented Sep 5, 2015

I'm currently involved in setting up a Jenkins-based build for our project and I've found that the popular Git plugin for Jenkins takes the slightly weird approach of git init+fetch instead of git clone when setting up the repo on agent machines.

This appears to break Git LFS as all the binary files are still only the reference pointer files.

The plugin does not currently appear to have any advanced options to force git cloning and there are a number of aged active tickets for the plugin to support it.

The author has stated that adding git clone as a problem:

".... Git fetch was chosen because there are fewer ways it can hang (prompting for authentication information) in the various authentication scenarios supported by the plugin. It is even more challenging because there are currently no unit tests for the authentication scenarios, so they must be tested interactively (or not tested) at each plugin release. Adding the option to use "clone" instead of "fetch" would effectively double the cases we need to test in an already very complicated portion of the code."

I'm no expert to Git and don't propose any best solution to this problem, nor do I imply that the solution is one to be solved by Git LFS. However I'd like to seek advice here on what the best practice solution might be from the Git LFS point of view.

@technoweenie

This comment has been minimized.

Copy link
Member

commented Sep 5, 2015

In the upcoming Git LFS v0.6.0, you can run git lfs pull to download the lfs objects and replace the pointers. Does the git plugin let you add another command like that?

@strich

This comment has been minimized.

Copy link
Contributor Author

commented Sep 5, 2015

I believe that can be done yes. I'm also willing to contribute to the plugin to more officially support Git LFS by way of actual flags/options.

We could potentially have the plugin use lfs wrapper commands such as git lfs init/fetch. Would that be a better approach in your opinion?

@strich

This comment has been minimized.

Copy link
Contributor Author

commented Sep 5, 2015

@technoweenie Doesn't look like there is a simple way for a user to add custom git commands as part of the Git plugin process sadly. I can add a custom build step, for now. But I'm currently struggling to have any git-lfs command (Windows here) perform non-interactive auth.

Am I right in assuming that if I manually clone the Git LFS active repo the first time it will be fine with git fetch commands in future?

@technoweenie

This comment has been minimized.

Copy link
Member

commented Sep 5, 2015

I'm not sure the git plugin should have explicit Git LFS support. But it should be possible to change or add on to it's fetching routine.

Since CI servers typically don't commit, you don't actually need git lfs init. You can just run git lfs pull.

Note: the pull command doesn't exist in Git LFS v0.5.x. I'm hoping to push out v0.6.0 within the next week though. It's a huge improvement.

@strich

This comment has been minimized.

Copy link
Contributor Author

commented Sep 5, 2015

Normally I would agree, but the problem is that the Git plugin fully manages the Git login credentials during the build process and wipes them once its own build step is complete. If I try to add any git commands outside of that plugin then I'll have to go through a lot just to get auth working.

If you're confident git lfs pull is due within that time frame then that could be the best solution. I can write an additive sub-build step for the plugin that only those who want LFS can use.

@strich

This comment has been minimized.

Copy link
Contributor Author

commented Sep 12, 2015

So after downloading Git LFS v0.60 I tried adding Git LFS pull to the commands I run as part of the batch however I get errors. See issue #660.

@juddgaddie

This comment has been minimized.

Copy link

commented Mar 4, 2016

I'm seeing a similar problem with Jenkins I tried to do a git lfs pull, but get the following error: http://pastebin.com/c5h1yEgF

@matthauck

This comment has been minimized.

Copy link

commented Jun 14, 2016

Since this thread is mostly about jenkins usage of git lfs, and since it has such google presence for search terms "git lfs jenkins", I wanted to just note here I filed an issue with JIRA to add explicit support for git lfs, with some pull requests of how it could basically work. https://issues.jenkins-ci.org/browse/JENKINS-35687.

@strich

This comment has been minimized.

Copy link
Contributor Author

commented Jun 14, 2016

Hi @matthauck. If you haven't seen it already, there is an existing Jenkins issue I made many months ago with some discussion that might be relevant to your work: https://issues.jenkins-ci.org/browse/JENKINS-30318

@matthauck

This comment has been minimized.

Copy link

commented Jun 14, 2016

Yeah, saw that - I think I linked the issues in jira. Decided to not pursue that discussion since the git plugin seems pretty committed to init+fetch, and also for that to magically work with lfs, we would have have to run git lfs install as the jenkins user on all the jenkins nodes -- more setup steps == bad. So, decided to start a new issue for explicit support.

@qrilka

This comment has been minimized.

Copy link

commented Aug 17, 2016

@strich this issue seems to be almost a year old already - did you find any decent workaround maybe?

@strich

This comment has been minimized.

Copy link
Contributor Author

commented Aug 17, 2016

Nope! I use a second script step to run the correct commands after Git has gone.

@technoweenie

This comment has been minimized.

Copy link
Member

commented Aug 17, 2016

Closing this, as it's really about getting Jenkins to recognize Git LFS. As @strich notes, it can be worked around.

@pwr22

This comment has been minimized.

Copy link

commented Aug 26, 2016

Does anyone have an example workaround available somewhere?

@qrilka

This comment has been minimized.

Copy link

commented Aug 26, 2016

@pwr22 we've just added installation of git lfs and git lfs pull into job scripts, nothing fancy

@matthauck

This comment has been minimized.

Copy link

commented Aug 26, 2016

I referenced some PR's I created to get this working on a fork of the github plugin: https://issues.jenkins-ci.org/browse/JENKINS-35687

@vlseattlebot

This comment has been minimized.

Copy link

commented Aug 29, 2016

This is completely broken behavior for Jenkins not to support Git-LFS. It's incredible to me that the most popular build harness doesn't support such near-standard functionality. Anyone working on a project of any reasonable size, and particularly projects that do have a number of binaries in the repo, should be using LFS.

@lgreenlee

This comment has been minimized.

Copy link

commented Sep 25, 2016

@vlseattlebot in their defense, git lfs is not really that easy to use. If git pull worked liked they said it would then we wouldn't have these problems.

@qrilka

This comment has been minimized.

Copy link

commented Sep 25, 2016

@lgreenlee Jenkins uses init+fetch, why do you talk about git pull?

@creste

This comment has been minimized.

Copy link
Contributor

commented Dec 29, 2016

I created two new PRs that fixed a few issues with @matthauck's PRs:

@MarkEWaite

This comment has been minimized.

Copy link

commented Jan 21, 2017

I've started the evaluation of the two pull requests for the git client plugin and the git plugin. Notes are being added to jenkinsci/git-plugin#463

@maxter2323

This comment has been minimized.

Copy link

commented Feb 21, 2017

Hello @MarkEWaite . First of all I would like to say that I really appreciate your work and effort for the LFS support and Jenkins development. You are helping a lot of people.

At my work place I tried to set up Jenkins on Mac Os Sierra and encountered this LFS issue("error code 128" and "unknown command git-lfs").
To fix it, I did many things, including setting up a git path in Jenkins configuration, downloading your docker plugin snaphots, using experimental plugin versions and etc. All that time "git lfs" was working in the terminal for jenkins user and my account as well but failed in Jenkins pipeline building.

It even worked with:

sudo su jenkins
git lfs

For multiple times I was checking "usr/bin" and "usr/local/bin" to ensure that git-lfs is located there.
Everything seemed fine but the error was constantly occurring.

So when I decided to do this:
git --exec-path

I was surprised that instead of "/usr/bin/" or "/usr/local/bin" I see:
/Applications/Xcode.app/Contents/Developer/usr/libexec/git-core

So after this I just dragged "git-lfs" unix executable to this folder and now everything works.
I hope it will help somebody as it helped me.

@MarkEWaite

This comment has been minimized.

Copy link

commented Mar 28, 2017

The Jenkins git plugin 3.1.0 release, 4 Mar 2017, includes git large file support.

@aufano

This comment has been minimized.

Copy link

commented Jul 7, 2017

Now works!, thanks to @maxter2323 I had the same issue with jenkins installed in mac os , with git plugin 3.3.1

@ludddd

This comment has been minimized.

Copy link

commented Aug 9, 2017

@maxter2323 , great thanks!!!
For those who stumble upon this thread:
This fix (3 messages up) solves the problem with "git-lfs filter-process: git-lfs: command not found" also.

@alexdmotoc

This comment has been minimized.

Copy link

commented Mar 14, 2018

EDIT (fixed):
Homebrew installs git-lfs in /usr/local/bin; Jenkins must know about this.

I added the PATH environment variable to Manage Jenkins -> Nodes -> Configure -> Environment Vars and it works now

Deleted the modifications I mentioned below


Hi @maxter2323,

I also have Jenkins running on a macOS High Sierra, and had this issue, so tried your proposed solution.

My git --exec-path returned /Library/Developer/CommandLineTools/usr/libexec/git-core so I copied an alias to git-lfs there.

Jenkins does see git-lfs now, but I get this error:
stderr: git-lfs filter-process: /Library/Developer/CommandLineTools/usr/libexec/git-core/git-lfs: Permission denied fatal: The remote end hung up unexpectedly

Any suggestions what can I do in this case?

@matthauck

This comment has been minimized.

Copy link

commented Mar 14, 2018

Is there a way to lock this issue?

@git-lfs git-lfs locked as off topic and limited conversation to collaborators Mar 15, 2018

@ttaylorr

This comment has been minimized.

Copy link
Member

commented Mar 15, 2018

As @matthauck notes, this conversation is getting off-topic, since the issue was closed some time ago. If anyone is having issues with Git LFS that are different to the one described in the original body of the issue, then please do not hesitate to open a new issue stating so.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
You can’t perform that action at this time.