LuceneSearcher + multiprocessing problem

[davidoterof]

Hi,

I'm using LuceneSearcher in a multiprocessing setting, but it is not working as I expected. I crafted the following script to showcase the problem that I have:

from multiprocessing import Process
from pyserini.search.lucene import LuceneSearcher


def do_job(w):
    searcher = LuceneSearcher("...")
    print(w)
    print(searcher.doc(w).contents())
    searcher.close()

processes = []

# creating processes
for w in range(4):
    p = Process(target=do_job, args=(w,))
    processes.append(p)
    p.start()

# completing process
for p in processes:
    p.join()

When running this, it gives a bunch of errors like this:

jnius.JavaException: JVM exception occurred: org/apache/lucene/store/Lock java.lang.NoClassDefFoundError

And every time that I run it, the class name changes without any apparent pattern. I don't know if I'm misunderstanding something, and this is normal behaviour, or the problem is on pyserini's side.

thanks