Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JENKINS-35687] Add simple git lfs support #414

Closed
wants to merge 1 commit into from

Conversation

@matthauck
Copy link

commented Jun 14, 2016

Note: I had to do some minor changes to pom.xml to make this build with changes for jenkinsci/git-client-plugin#206, so it may well not pass the CI build...

@MarkEWaite

This comment has been minimized.

Copy link

commented Jun 14, 2016

How does the user tell the plugin to enable LFS support for a specific job? I don't see any user interface change in the plugin and can't understand what calls this new code if there is no user interface enabling it for the job.

Ah, I wasn't attentive enough. I assume that because it is a GitSCMExtension, the "Enable Git LFS support" will appear as a drop down choice in the "Additional Behaviours" section.

@matthauck

This comment has been minimized.

Copy link
Author

commented Jun 14, 2016

Yeah, was just about to add that it is an additional behavior. I can add a help.html, however, which would probably make it more useful.

@HRMPW

This comment has been minimized.

Copy link

commented Aug 11, 2016

Is this stalled or something that might still make it into the git plugin. Also, the most recent version of jgit has support for lfs but it doesn't look like it is being pulled in.

@MarkEWaite

This comment has been minimized.

Copy link

commented Aug 11, 2016

It is something that might still make it into the git plugin, but not before my hoped September 2016 release of git plugin 3.0.0 and git client plugin 2.0.0. Those releases are planned to be in time for Jenkins World 2016.

Git plugin 3.0.0 adds submodule authentication support and drops support for Java 6. It requires at least Jenkins 1.625. It has been in beta test in various incarnations for almost a year.

Git client plugin 2.0.0 adds submodule authentication support, drops support for Java 6, and upgrades the JGit implementation from JGit 3 to JGit 4. It requires at least Jenkins 1.625. It has been in beta test in various incarnations for almost a year.

The git client plugin release is particularly substantial because the JGit API changes from JGit 3 to JGit 4 will break users of plugins which depend on the JGit implementation provided by the git client plugin. You can read the jenkins-developers mailing list posting for more details.

@matthauck

This comment has been minimized.

Copy link
Author

commented Aug 15, 2016

Glad to hear that lfs support may yet have a future here! 😅

@HRMPW

This comment has been minimized.

Copy link

commented Oct 5, 2016

@MarkEWaite I hope JW went well for you. There was quite a of activity at the booth. Now that 3.0 has been released I thought I might ask about LFS support again. I have had several people ask me about this feature in Jenkins.

@MarkEWaite

This comment has been minimized.

Copy link

commented Oct 5, 2016

I have several other reviews in the queue before this one, so it will likely be a month or more before the reviews are complete. One Jira user has started evaluating this pull request (and the matching pull request for the git client plugin). If others can do the same evaluation of a pre-release build, it makes me more confident in the change, and reduces the amount of verification that I need to do.

@pyther

This comment has been minimized.

Copy link

commented Nov 3, 2016

I am extremely interested in this pull request. git-lfs support is Jenkins is something that my organization needs like yesterday. I attempted to build both the git plugin with a patched git-client plugin, but got build errors that I could not figure out. I'd be glad to test on my instance if I can get some help building the plugin.

@MarkEWaite

This comment has been minimized.

Copy link

commented Nov 4, 2016

I've built a version of the git client plugin which includes the proposed lfs change and a version of the git plugin which includes the proposed change. Since I'm not a git lfs user, I'll need to rely on others to decide if the functionality is sufficiently rich to justify including it in the plugin

@zoomequipd

This comment has been minimized.

Copy link

commented Nov 13, 2016

Having been suffering from builds hanging (i assume waiting for auth) in LFS enabled repos -
I pulled down and installed the git-client and git plugin linked to by @MarkEWaite [https://github.com//pull/414#issuecomment-258318580]

added the "GIT LFS pull after checkout" option and it errored out - it is this actually trying to do a git lfs pull before it does the checkout?

00:13:54 Started by user Brandon Murphy
00:13:54 Building in workspace /var/lib/jenkins/workspace/website
00:13:54 [WS-CLEANUP] Deleting project workspace...
00:13:54 [WS-CLEANUP] Done
00:13:54 Cloning the remote Git repository
00:13:54 Using shallow clone
00:13:54 Cloning repository https://bitbucket.org/secdsm/website.git
00:13:54  > git init /var/lib/jenkins/workspace/website # timeout=10
00:13:54 Fetching upstream changes from https://bitbucket.org/secdsm/website.git
00:13:54  > git --version # timeout=10
00:13:54 using GIT_ASKPASS to set credentials 
00:13:54  > git fetch --tags --progress https://bitbucket.org/secdsm/website.git +refs/heads/*:refs/remotes/origin/* --depth=1
00:13:57  > git config remote.origin.url https://bitbucket.org/secdsm/website.git # timeout=10
00:13:57  > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
00:13:57  > git config remote.origin.url https://bitbucket.org/secdsm/website.git # timeout=10
00:13:57 Enabling Git LFS pull
00:13:57 Fetching upstream changes from https://bitbucket.org/secdsm/website.git
00:13:57 using GIT_ASKPASS to set credentials 
00:13:57  > git fetch --tags --progress https://bitbucket.org/secdsm/website.git +refs/heads/*:refs/remotes/origin/* --depth=1
00:14:00 using GIT_ASKPASS to set credentials 
00:14:00  > git lfs pull
00:14:00 ERROR: Error fetching remote repo 'origin'
00:14:00 hudson.plugins.git.GitException: Failed to fetch from https://bitbucket.org/secdsm/website.git
00:14:00    at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:803)
00:14:00    at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1063)
00:14:00    at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1094)
00:14:00    at hudson.scm.SCM.checkout(SCM.java:495)
00:14:00    at hudson.model.AbstractProject.checkout(AbstractProject.java:1278)
00:14:00    at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:604)
00:14:00    at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
00:14:00    at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529)
00:14:00    at hudson.model.Run.execute(Run.java:1720)
00:14:00    at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
00:14:00    at hudson.model.ResourceController.execute(ResourceController.java:98)
00:14:00    at hudson.model.Executor.run(Executor.java:404)
00:14:00 Caused by: hudson.plugins.git.GitException: Command "git lfs pull" returned status code 2:
00:14:00 stdout: 
00:14:00 stderr: Could not pull
00:14:00 
00:14:00 Errors logged to /var/lib/jenkins/workspace/website/.git/lfs/objects/logs/20161113T001400.052039.log
00:14:00 Use `git lfs logs last` to view the log.
00:14:00 
00:14:00    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1758)
00:14:00    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1502)
00:14:00    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:64)
00:14:00    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:327)
00:14:00    at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:801)
00:14:00    ... 11 more
00:14:00 ERROR: null
00:14:00 [WS-CLEANUP] Deleting project workspace...[WS-CLEANUP] Skipped based on build state FAILURE
00:14:00 Finished: FAILURE

`

the log from 'git lfs logs last' shows


git-lfs/1.4.4 (GitHub; linux amd64; go 1.7.3)
git version 2.7.4

$ git-lfs pull
Could not pull

Git can't resolve ref: "HEAD"

ENV:
[snip]

@matthauck

This comment has been minimized.

Copy link
Author

commented Nov 14, 2016

Hmm. Interesting, it should be doing the lfs pull after the checkout so that it pulls the files for the right branch...

@estyrke

This comment has been minimized.

Copy link

commented Nov 30, 2016

@matthauck Since the decorator is added on the fetch command I guess it will happen before the checkout. Would it be better to do a two-step process of adding "git lfs fetch" to the Fetch command and "git lfs checkout" to the Checkout command? It seems like it would solve this problem and also looks more symmetrical to me.

I made a try at this, working from your existing modifications:
jenkinsci/git-client-plugin@master...estyrke:git-lfs
master...estyrke:git-lfs

However, I did not manage to get a build environment working, so these changes are completely untested. Feel free to use them if you like.

@zoomequipd

This comment has been minimized.

Copy link

commented Dec 13, 2016

@estyrke - If you can compile the packages into a SNAPSHOT* for me, (sorry, I'm clueless how to do that) I can test it out.
EDIT - called it a SNAPSHOT

@matthauck

This comment has been minimized.

Copy link
Author

commented Dec 15, 2016

@estyrke, I'm not entirely available at present to test that, we have something working now that I don't really want to / can change rihgt now. But your approach seems like the right way to do it. I'd suggest opening a PR and going with that solution? We can just close this one.

@MarkEWaite

This comment has been minimized.

Copy link

commented Jan 21, 2017

I'm closing this pull request so that I can focus my evaluation efforts on #463 .

@MarkEWaite MarkEWaite closed this Jan 21, 2017

@matthauck matthauck deleted the matthauck:JENKINS-35687 branch Jan 23, 2017

@matthauck

This comment has been minimized.

Copy link
Author

commented Jan 23, 2017

👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.