[JENKINS-21248] Support shallow submodule update

[fujii]

Fix for https://issues.jenkins-ci.org/browse/JENKINS-21248

[fujii]

This patch is the git-plugin part of this fix.
fujii/git-plugin@035090e

[MarkEWaite]

Please provide tests for the shallow setting on submodule clone. The existing submodule tests should be a good starting point to add further tests.

[fujii]

Thank you for your feedback, @MarkEWaite.
I created a test, and it passes on my env. But it fails in CI env.
What makes the difference? CI server can access github.com during tests?

[MarkEWaite]

I see the same failure when I execute:

mkdir tmp
cd tmp
git clone https://github.com/jenkinsci/git-client-plugin
cd git-client-plugin
git checkout -b fujii-shallow-submodule-update master
git pull https://github.com/fujii/git-client-plugin.git shallow-submodule-update
mvn clean -Dtest=CliGitAPIImplTest test

I assumed it was the missing submoduleInit() call in the new test compared to the test which precedes it in the file. Unfortunately, that was not enough to fix the test. I think you'll need to investigate further.

[fujii]

Thank you for testing my patch, @MarkEWaite.
Unfortunately, I can't reproduce the error with your procedure.
I guess it might depend on the region.
So, I created a new test fetching a submodule from the local mirror target/clone.git.
This patch will fail in the CI auto test because https://github.com/jenkinsci/git-client-plugin.git doesn't have tests/getRelativeSubmodule branch yet.
It is in https://github.com/fujii/git-client-plugin/tree/tests/getRelativeSubmodule
How do you think this idea?

[darxriggs]

Hi @fujii, could you rebase this pull request on the current master branch?

[darxriggs]

If possible I would go for int instead of Integer for the method parameter and instance variable. (I also see boolean and no Boolean).

[darxriggs]

Ok. I have seen now that this is done like this to be consisent with the methods for clone/fetch.

[darxriggs]

Other test cases would also be useful:

• no shallow(...) call
• shallow(false)
• depth(...)

[fujii]

@darxriggs: Thank you. I will do it. But, I'm pretty busy now. Feel free to take over.

[darxriggs]

Hi @fujii, the error you mentioned, is it about referencing shas on github when cloning? Github does not support this (see isaacs/github#436). The respective git option would be uploadpack.allowReachableSHA1InWant. The error output I get is as follows:

hudson.plugins.git.GitException: Command "git submodule update --depth=1 modules/sshkeys" returned status code 1:
stdout: 
stderr: Cloning into '/tmp/jenkinsTests.tmp/jenkins3443046290638298345test/modules/sshkeys'...
error: Server does not allow request for unadvertised object 689c45ed57f0829735f9a2b16760c14236fe21d9
Fetched in submodule path 'modules/sshkeys', but it did not contain 689c45ed57f0829735f9a2b16760c14236fe21d9. Direct fetching of that commit failed.

Your proposed solution with an extra branch that contains a submodule referencing a local git repository is a clever trick. But I find it a bit confusing and an extra branch has to be maintained.

Another idea is to put a repository with submodules into src/test/resources (there are already others) that contains all relevant properties to test submodules. This would also reduce the need to clone submodules from github which implies some risk and constraints.

What are your thoughts @MarkEWaite?

[MarkEWaite]

Adding a binary copy of a specific repository into src/test/resources would be fine. I have no problem with extra branches in the git-client-plugin repository for specific tests. I hesitate to have those branches depend on something outside the repository, unless the tests which use those branches are entirely optional.

[darxriggs]

Hi @MarkEWaite, should using shallow for JGit throw this UnsupportedOperationException here or just log a WARNING when calling shallow(true) at lines 2224-2227?

As a reference FetchCommand and CloneCommand just perform logging. That's not consistent in this class.

[MarkEWaite]

You're right @darxriggs , it would be good enough to log the implementation gap rather than fail the checkout. Shallow clone is a performance optimization that isn't critical to the typical use of the repository.

[darxriggs]

As I cannot update this one, I prepared a new pull request #344 with improvements.

[fujii]

Thanks. Closed this ticket.