Performance issue

[borisbaldassari]

Hi good people,

I've setup a sherlock instance on a quite large codebase at [1]. The indexes took several days to build, it worked well, but now the interface is so slow that it can hardly be used [2].

Is it considered normal, or can I do anything to improve the speed of the UI? Is it just the host's perfs (vm with 8 procs and 16GB of RAM)?

I could not find anything about perfs on the wiki, and cannot find a mailing list to ask questions on. Hence this information request. Thanks in advance, have a wonderful end of year! :-)

[1] http://ci3.castalia.camp:7777/
[2] http://ci3.castalia.camp:7777/search?q=IFrame

--
boris

[cbess]

Are you using the Xapian backend?

There is a small mention about performance considerations in the README under the Basic Setup and End-to-end Interface.

Basically review the two available backends.

Let me know if that helps.

[borisbaldassari]

Hi Christopher,

Thanks for the quick reply! I'm using the default backend indeed.
I'll give xapian a try and let you know asap..

Thanks! (and a happy new year,btw! :-)

[borisbaldassari]

I can't get the xapian backend working, I'm sorry.

Xapian eventually installed ok, sherlock actually tries to use it, but the db cannot be opened:

  File "/home/boris/text-sherlock/core/sherlock/backends/xapian_backend.py", line 63, in open_index
    self.index = xapian.WritableDatabase(path, xapian.DB_OPEN)
  File "/home/boris/text-sherlock/sherlock_env/lib/python3.4/site-packages/xapian/__init__.py", line 9151, in __init__
    _xapian.WritableDatabase_swiginit(self, _xapian.new_WritableDatabase(*args))
xapian.DatabaseOpeningError: /home/boris/text-sherlock/data/indexes/main/iamglass: Failed to open glass revision file for reading (No such file or directory)

I've added the following lines to local_settings.yml:

default_searcher: 'xapian'
default_indexer: 'xapian'
index_paths: '/crossminer/git_repos/'

Hum.. The index_paths directive doesn't seem to work either, but that's another problem I guess.

Any hint? Thanks a lot in advance to help me get that up and running..

[borisbaldassari]

The folder /home/boris/text-sherlock/data/indexes/main/ exists, is writeable by the user. The file iamglass doesn't exist, though.

[borisbaldassari]

My setup is probably too big, closing the issue for clarity's sake. Thanks for the support team, you're great! :-)