Missing blob when indexing a repository with a submodule

[gitblit]

Originally reported on Google Code with ID 119

What steps will reproduce the problem?
1. Setup gitblit and push the contents of a specific git repository (unfortunately
with proprietary contents) into a repository managed by gitblit
2. Switch on indexing in one of the branches of this repository

What is the expected output? What do you see instead?

Expected: The branch should be searchable and no error messages should appear in the
log.
Actual: We get the following message in the log:

ERROR Exception while reindexing foo.git
org.eclipse.jgit.errors.MissingObjectException: Missing blob 4bd2ec7f20cda43a03aa6d190a9313727383339d
    at org.eclipse.jgit.storage.file.WindowCursor.open(WindowCursor.java:126)
    at org.eclipse.jgit.lib.ObjectDatabase.open(ObjectDatabase.java:176)
    at org.eclipse.jgit.lib.Repository.open(Repository.java:273)
    at com.gitblit.LuceneExecutor.reindex(LuceneExecutor.java:557)
    at com.gitblit.LuceneExecutor.index(LuceneExecutor.java:193)
    at com.gitblit.LuceneExecutor.run(LuceneExecutor.java:173)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:679)
ERROR Could not build foo.git Lucene index!


What version of the product are you using? On what operating system?
Gitblit: 1.0.0
OS: CentOS 6.2


Please provide any additional information below.
The repository size is 226MB. I have started java with "-Xmx10240M".

I have tried to do the same with other repositories without any problems. Unfortunately
I cannot shared the repository.

Reported by peter@peca.dk on 2012-08-05 20:37:45

[gitblit]

Let's track down this blob.  If you run the following command on the source repository,
do you get a result?

git ls-files --stage | grep 4bd2ec7f20cda43a03aa6d190a9313727383339d

Can you repeat the same test on the Gitblit copy of the repository?  Do you get the
same result?

Reported by James.Moger on 2012-08-06 13:16:14

[gitblit]

I thought I might have forgotten where the initial push was done from. In order to remove
this uncertainty I setup a test gitblit on my development machine, and repeated the
steps.

This time I pushed from a non-bare repository. I got the following output when running
the command you suggested on the development machine:

160000 c39e9d6693c303db7d9969b1196bfb35fad79452 0   foo-web-client/bar

The "foo-web-client/bar" directory is a git submodule. I guess the existence of a git
submodule is what differentiates our real repository from the tests I have done. The
"git ls-files --stage" command still does not give any output on the repository handled
by Gitblit.

Is the indexing supposed to work with submodules?

Reported by peter@peca.dk on 2012-08-06 13:48:42

[gitblit]

Ah.  That is likely the problem.  I had not considered submodules and I have 0 experience
with them.

Reported by James.Moger on 2012-08-06 14:13:38

[gitblit]

Reported by James.Moger on 2012-08-06 21:44:16

[gitblit]

I think the best way to handle git submodules is to ignore them. They are separate repositories
and can be added as repositories in gitblit the same way that the repository referencing
them has been added.

If you agree with this, the issue can be fixed by checking the mode of the objects
being added to the TreeMap of objects to be indexed, and ignore it if the mode is FileMode.GITLINK.

I have made this change in this repository: http://code.google.com/r/peter-issue415/.
(I don't know if this is the preferred way of sending patches to the project).

It seems to be working in my case.

Reported by peter@peca.dk on 2012-08-07 11:15:57

[gitblit]

That sounds reasonable.  I merged your commit and also took care of the other indexing
case.  Thanks for your contribution!  Sometimes I shake my head when I receive actual
patch files instead of a link to someone's clone which I can pull from - afterall this
is git?!  I did not realize GoogleCode offered personal clones, I guess they are feeling
the pressure of GitHub.

Reported by James.Moger on 2012-08-07 12:18:13

• Status changed: Queued
• Labels added: Milestone-1.0.1

[gitblit]

Reported by James.Moger on 2012-08-20 02:06:35

• Labels added: Milestone-1.1.0
• Labels removed: Milestone-1.0.1

[gitblit]

Fix/change released in 1.1.0.

Reported by James.Moger on 2012-08-25 12:20:42

• Status changed: Fixed