Hebrew support

[Rashty]

Hi,
Is there a way to search text in Hebrew?

[alex-money]

@Rashty what error are you getting?

[olivernn]

The lunr-languages project has support for many languages, I don't think that hebrew is one of them, though it should at least give you some ideas as to what is involved in adding support for another language.

The language specific parts are the tokenisation/trimming, stemming and stop-word filters. The default implementation of these are english specific, and will probably be causing some issues with a language such as Hebrew.

The simplest approach would be to remove the existing english specific pipeline functions before you add any documents to the index.

idx.pipeline.reset()

The more involved approach would be to implement these text processing functions specifically for Hebrew, take a look at the lunr-languages project for some implementations for other languages for ideas. If you do go this route, please do update this issue with your progress so others can also benefit from being able to use lunr with Hebrew.

[biodranik]

Has anyone implemented Hebrew support for lunr? Please share your implementation.

[olivernn]

Found this just now, might be at least a starting point for anyone who is willing to do the work required to implement this.

https://github.com/iddoberger/awesome-hebrew-nlp