Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JENKINS-35687] Add simple git lfs support #463

Merged
merged 1 commit into from Feb 27, 2017

Conversation

@creste
Copy link

commented Dec 29, 2016

This is a refactoring of @matthauck's changes from #414. This PR depends on jenkinsci/git-client-plugin#232, which explains the changes in more detail.

@creste creste force-pushed the creste:JENKINS-35687 branch from 233b76a to 53594de Jan 4, 2017

@creste creste force-pushed the creste:JENKINS-35687 branch from 53594de to a809052 Jan 4, 2017

@newsoulbr

This comment has been minimized.

Copy link

commented Jan 5, 2017

I added this code on my local repository based on master and installed in my environment and it worked fine, no issues at this moment. Definitely fixed my problem with Git LFS

@baffles

This comment has been minimized.

Copy link
Member

commented Jan 11, 2017

FWIW, I've built the modified plugin and it seems to be working fine for me (we needed LFS support within Jenkins).

@johnwbyrd

This comment has been minimized.

Copy link

commented Jan 17, 2017

Hey baffles, what is your take on the failed checks?

@MarkEWaite

This comment has been minimized.

Copy link

commented Jan 17, 2017

The LFS support in the git-plugin depends on LFS related changes in the git-client-plugin. Since the job on ci.jenkins.io does not have a published copy (even snapshot) of the necessary git-client-plugin changes, it can't build (or test) the changes in the git plugin.

I believe that recent changes to artifactory repository permissions may now allow anyone to push a snapshot build to the jenkins artifactory, so it may be possible to resolve that. However, I don't think that is actually very much help to the pull request evaluation process.

When I start evaluating this pull request in detail, I'll run the automated tests on many different platforms (Windows, Red Hat 6, Red Hat 7, Debian 7, Debian 8, Ubuntu 14, Ubuntu 16, possibly others) and will run interactive tests using those same platforms. The automated tests on ci.jenkins.io are nice to have, but in this particular case, they are a small portion of the total work to evaluate the pull request.

@MarkEWaite

This comment has been minimized.

Copy link

commented Jan 21, 2017

I've added an unauthenticated verification job to my lts-with-plugins docker repository. That job passes with git client plugin PR232 and this pull request.

I've added an authenticated verification job to my private docker repo. That job passes with git client plugin PR232 and this pull request.

I've added an unauthenticated verification job with Jenkins pipeline to my jenkins-bugs public repository. That job passes with git client plugin PR232 and this pull request.

I've added an authenticated verification job with Jenkins pipeline and it fails to checkout with an error. The git lfs error message indicates that it is expecting git checkout to use authentication. The git lfs logging message that is written says that git checkout needs authentication because it is requesting a large file over https. I suspect that means the lfsRemote is somehow not set in the context where that checkout call is made.

The error message is generated by the git checkout command and says:

git-lfs/1.5.4 (GitHub; linux amd64; go 1.7.4; git 71b637f1)
git version 2.1.4

$ git-lfs smudge -- large-file.lfs
Error downloading object: large-file.lfs (9e7fee800da993ed289cebdf0e70797c3a4c6eb16483ea13e1999dd677f98346)

Smudge error: Error downloading 9e7fee800da993ed289cebdf0e70797c3a4c6eb16483ea13e1999dd677f98346: batch request: exit status 255: Permission denied (publickey).: batch request: exit status 255: Permission denied (publickey).
github.com/git-lfs/git-lfs/errors.newWrappedError
	/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/errors/types.go:166
github.com/git-lfs/git-lfs/errors.NewSmudgeError
	/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/errors/types.go:252
github.com/git-lfs/git-lfs/lfs.PointerSmudge
	/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/lfs/pointer_smudge.go:70
github.com/git-lfs/git-lfs/lfs.(*Pointer).Smudge
	/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/lfs/pointer.go:65
github.com/git-lfs/git-lfs/commands.smudge
	/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/commands/command_smudge.go:84
github.com/git-lfs/git-lfs/commands.smudgeCommand
	/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/commands/command_smudge.go:112
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).execute
	/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:477
github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra.(*Command).Execute
	/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/vendor/github.com/spf13/cobra/command.go:551
github.com/git-lfs/git-lfs/commands.Run
	/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/commands/run.go:66
main.main
	/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/git-lfs.go:33
runtime.main
	/usr/local/Cellar/go/1.7.4/libexec/src/runtime/proc.go:183
runtime.goexit
	/usr/local/Cellar/go/1.7.4/libexec/src/runtime/asm_amd64.s:2086

ENV:
LocalWorkingDir=/var/jenkins_home/workspace/-multi-branch_JENKINS-35687-SD7REQVKWPXKTYFWOLW3OX6GLUJMKBUP2UC5SSJJZZCY4QA752VA@script
LocalGitDir=/var/jenkins_home/workspace/-multi-branch_JENKINS-35687-SD7REQVKWPXKTYFWOLW3OX6GLUJMKBUP2UC5SSJJZZCY4QA752VA@script/.git
LocalGitStorageDir=/var/jenkins_home/workspace/-multi-branch_JENKINS-35687-SD7REQVKWPXKTYFWOLW3OX6GLUJMKBUP2UC5SSJJZZCY4QA752VA@script/.git
LocalMediaDir=/var/jenkins_home/workspace/-multi-branch_JENKINS-35687-SD7REQVKWPXKTYFWOLW3OX6GLUJMKBUP2UC5SSJJZZCY4QA752VA@script/.git/lfs/objects
LocalReferenceDir=
TempDir=/var/jenkins_home/workspace/-multi-branch_JENKINS-35687-SD7REQVKWPXKTYFWOLW3OX6GLUJMKBUP2UC5SSJJZZCY4QA752VA@script/.git/lfs/tmp
ConcurrentTransfers=3
TusTransfers=false
BasicTransfersOnly=false
BatchTransfer=true
SkipDownloadErrors=false
FetchRecentAlways=false
FetchRecentRefsDays=7
FetchRecentCommitsDays=0
FetchRecentRefsIncludeRemotes=true
PruneOffsetDays=3
PruneVerifyRemoteAlways=false
PruneRemoteName=origin
AccessDownload=none
AccessUpload=none
DownloadTransfers=basic
UploadTransfers=basic
GIT_ASKPASS=echo
GIT_COMMIT=270cdf3e59596743343f6c2ed2a1e3763bf0b213
GIT_DIR=.git
GIT_LOCAL_BRANCH=JENKINS-35687
GIT_PREFIX=
GIT_BRANCH=JENKINS-35687

More investigation will be needed.

@creste

This comment has been minimized.

Copy link
Author

commented Jan 21, 2017

I have successfully been using #463 in a test environment with pipeline builds for a while now. It looks like the error you are getting is caused by an authentication error with GitHub. I suspect either invalid or no credentials have been specified.

In my Multibranch Pipeline configuration I had to configure the credentials to use for Git on the web UI. I'm using public+private key authentication with my git server (not GitHub). In my pipeline script, I'm using the following code to clone the repository:

        checkout([
                $class: 'GitSCM',
                branches: scm.branches,
                browser: scm.browser,
                doGenerateSubmoduleConfigurations: scm.doGenerateSubmoduleConfigurations,
                extensions: scm.extensions + [
                        [ $class: 'GitLFSPull' ]
                ],
                submoduleCfg: scm.submoduleCfg,
                userRemoteConfigs: scm.userRemoteConfigs
        ])

I purposefully added the GitLFSPull extension in the pipeline script because I only want to pull large files under certain circumstances. The above command inherits the credentials configured in the UI so that I don't have to specify credentials in the pipeline script.

As you can see in CliGitAPIImpl.java, the git lfs pull command is being executed with the same credentials used for the git checkout command. If one works then the other one should work too, assuming the LFS remote is the same as the checkout remote. Although the remotes should be the same in all cases I can think of, if they were different then you have to make sure the same private key is trusted in both remotes.

Here are some ideas I have for troubleshooting the problem:

  • Check to see how many remotes are configured in the repository and what their URLs are
  • Try to manually run git lfs pull from the same repository and environment that Jenkins is running in.
  • Add GIT_TRACE=1 before calls to git to see what SSH command it being executed. You can then manually run the SSH command with -vvv to see which authentication methods and private key(s) are being used.
  • Try to locate the temporary file where Jenkins stores the private key during checkout and make sure it is the correct private key.
  • If your private key is password protected them make sure the password is correct.
@MarkEWaite

This comment has been minimized.

Copy link

commented Jan 22, 2017

Thanks for the suggestions. They resolved my issue.

I had configured the GitLFSPull class in the Jenkinsfile on the repository branch, but had failed to add it to the parent job definition in Jenkins. The setting was read from GitHub as expected, but the parent repository where the Jenkinsfile is extracted did not include the LFS extension, so was failing the checkout.

Once I added it to the parent job definition, the job ran successfully.

I think that means there are many different use cases to be evaluated, like:

  • LFS in a multi-configuration job across multiple platforms and git versions (already evaluated, though needs checks for both https and ssh)
  • LFS with multi-branch pipeline (with LFS enabled in the job definition - my failure case)
  • LFS in a single pipeline job (confirm pipeline syntax shows correctly, etc., both https and ssh, ...)
  • LFS in one branch of a repository, but not in a different branch (already being tested)
  • LFS with JGit (there are some hints it might work)

I don't think we need to resolve all those use cases, but I'd like to explore and understand them better before including the changes in a plugin release.

I also want to perform integration tests based on JENKINS-30318, JENKINS-35687, JENKINS-38708, and JENKINS-40174.

@creste

This comment has been minimized.

Copy link
Author

commented Jan 23, 2017

I'm glad to hear you resolved the issue! Please let me know if there is anything you'd like me do to assist with evaluating the other use cases.

@erebel55

This comment has been minimized.

Copy link

commented Feb 1, 2017

What is keeping this from being merged? The PR is pretty tiny, seems like important functionality to me. I'm having to copy over files manually after jenkins build has started, since it can't fetch the lfs files correctly.

@jhoblitt

This comment has been minimized.

Copy link
Member

commented Feb 1, 2017

@erebel55 It depends on changes in another plugin and LFS is difficult to test. https://github.com/jenkinsci/git-client-plugin/pull/232/files

@MarkEWaite

This comment has been minimized.

Copy link

commented Feb 2, 2017

I'd certainly like to have additional help testing the proposed changes. You can download the git client plugin and the git plugin from the lts-with-plugins branch of my docker instance.

I deeply appreciate comments on pull requests which say "I've installed this plugin and am using it successfully". I also deeply appreciate comments which indicate when a problem has been encountered.

With over 80 000 installations of the git plugin and git client plugin, I like to be careful about changes, especially additions of significant new capability (like large file support).

@gilramir

This comment has been minimized.

Copy link

commented Feb 9, 2017

I'm currently using the plugins and it's working for me.

@thetaiter

This comment has been minimized.

Copy link

commented Feb 10, 2017

For some reason the first pull does not work with the plugin. No matter how much time I allow for checkout by increasing the timeout, It waits forever and eventually times out. I have to go to the machine and manually pull with "git lfs clone ", after that, the plugin works fine.

@MarkEWaite

This comment has been minimized.

Copy link

commented Feb 10, 2017

@thetaiter, any chance you're using a non-default location as the source of your large files? The current implementation assumes that the large files are coming from the default source.

@thetaiter

This comment has been minimized.

Copy link

commented Feb 13, 2017

@MarkEWaite If you are referring to the Endpoint URL that I see what I run git lfs env, I believe that is the default location. We have not changed it. The repo as well as this Endpoint URL are on a private GHE server. The Endpoint URL is https://<GHE_URL>/<organization>/<repo>.git/info/lfs (auth=basic).

@MarkEWaite MarkEWaite merged commit ae6883b into jenkinsci:master Feb 27, 2017

0 of 2 checks passed

Jenkins Looks like there's a problem with this pull request
Details
continuous-integration/jenkins/pr-merge This commit cannot be built
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
10 participants
You can’t perform that action at this time.