Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing blob when indexing a repository with a submodule #415

Closed
gitblit opened this issue Aug 12, 2015 · 8 comments
Closed

Missing blob when indexing a repository with a submodule #415

gitblit opened this issue Aug 12, 2015 · 8 comments

Comments

@gitblit
Copy link
Owner

@gitblit gitblit commented Aug 12, 2015

Originally reported on Google Code with ID 119

What steps will reproduce the problem?
1. Setup gitblit and push the contents of a specific git repository (unfortunately
with proprietary contents) into a repository managed by gitblit
2. Switch on indexing in one of the branches of this repository

What is the expected output? What do you see instead?

Expected: The branch should be searchable and no error messages should appear in the
log.
Actual: We get the following message in the log:

ERROR Exception while reindexing foo.git
org.eclipse.jgit.errors.MissingObjectException: Missing blob 4bd2ec7f20cda43a03aa6d190a9313727383339d
    at org.eclipse.jgit.storage.file.WindowCursor.open(WindowCursor.java:126)
    at org.eclipse.jgit.lib.ObjectDatabase.open(ObjectDatabase.java:176)
    at org.eclipse.jgit.lib.Repository.open(Repository.java:273)
    at com.gitblit.LuceneExecutor.reindex(LuceneExecutor.java:557)
    at com.gitblit.LuceneExecutor.index(LuceneExecutor.java:193)
    at com.gitblit.LuceneExecutor.run(LuceneExecutor.java:173)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:679)
ERROR Could not build foo.git Lucene index!


What version of the product are you using? On what operating system?
Gitblit: 1.0.0
OS: CentOS 6.2


Please provide any additional information below.
The repository size is 226MB. I have started java with "-Xmx10240M".

I have tried to do the same with other repositories without any problems. Unfortunately
I cannot shared the repository.

Reported by peter@peca.dk on 2012-08-05 20:37:45

@gitblit
Copy link
Owner Author

@gitblit gitblit commented Aug 12, 2015

Let's track down this blob.  If you run the following command on the source repository,
do you get a result?

git ls-files --stage | grep 4bd2ec7f20cda43a03aa6d190a9313727383339d

Can you repeat the same test on the Gitblit copy of the repository?  Do you get the
same result?

Reported by James.Moger on 2012-08-06 13:16:14

@gitblit
Copy link
Owner Author

@gitblit gitblit commented Aug 12, 2015

I thought I might have forgotten where the initial push was done from. In order to remove
this uncertainty I setup a test gitblit on my development machine, and repeated the
steps.

This time I pushed from a non-bare repository. I got the following output when running
the command you suggested on the development machine:

160000 c39e9d6693c303db7d9969b1196bfb35fad79452 0   foo-web-client/bar

The "foo-web-client/bar" directory is a git submodule. I guess the existence of a git
submodule is what differentiates our real repository from the tests I have done. The
"git ls-files --stage" command still does not give any output on the repository handled
by Gitblit.

Is the indexing supposed to work with submodules?

Reported by peter@peca.dk on 2012-08-06 13:48:42

@gitblit
Copy link
Owner Author

@gitblit gitblit commented Aug 12, 2015

Ah.  That is likely the problem.  I had not considered submodules and I have 0 experience
with them.

Reported by James.Moger on 2012-08-06 14:13:38

@gitblit
Copy link
Owner Author

@gitblit gitblit commented Aug 12, 2015

Reported by James.Moger on 2012-08-06 21:44:16

@gitblit
Copy link
Owner Author

@gitblit gitblit commented Aug 12, 2015

I think the best way to handle git submodules is to ignore them. They are separate repositories
and can be added as repositories in gitblit the same way that the repository referencing
them has been added.

If you agree with this, the issue can be fixed by checking the mode of the objects
being added to the TreeMap of objects to be indexed, and ignore it if the mode is FileMode.GITLINK.

I have made this change in this repository: http://code.google.com/r/peter-issue415/.
(I don't know if this is the preferred way of sending patches to the project).

It seems to be working in my case.

Reported by peter@peca.dk on 2012-08-07 11:15:57

@gitblit
Copy link
Owner Author

@gitblit gitblit commented Aug 12, 2015

That sounds reasonable.  I merged your commit and also took care of the other indexing
case.  Thanks for your contribution!  Sometimes I shake my head when I receive actual
patch files instead of a link to someone's clone which I can pull from - afterall this
is git?!  I did not realize GoogleCode offered personal clones, I guess they are feeling
the pressure of GitHub.

Reported by James.Moger on 2012-08-07 12:18:13

  • Status changed: Queued
  • Labels added: Milestone-1.0.1

@gitblit
Copy link
Owner Author

@gitblit gitblit commented Aug 12, 2015

Reported by James.Moger on 2012-08-20 02:06:35

  • Labels added: Milestone-1.1.0
  • Labels removed: Milestone-1.0.1

@gitblit
Copy link
Owner Author

@gitblit gitblit commented Aug 12, 2015

Fix/change released in 1.1.0.

Reported by James.Moger on 2012-08-25 12:20:42

  • Status changed: Fixed

@gitblit gitblit closed this as completed Aug 12, 2015
@flaix flaix added this to the 1.1.0 milestone Dec 13, 2016
@flaix flaix added this to the 1.1.0 milestone Dec 13, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants