No results found error when searching in Japanese

[hieunv5501]

I am using Meilisearch for japanese data. In my data there is a sentence that is : 日東電工株式会社の経営者 .
I'm having a problem that when I search with words like: 日東, 日東電工, 日東電工株式会社. I can't find any results.

Data

Search with words like: 日東, 日東電工, 日東電工株式会社

I am in need of a solution to this or a version that fixes this.

Meilisearch version: v0.29.3

[oluademola]

Hello @hieunv5501. Sorry for that. It could be an issue with the Japanese tokenizer. However, your version of meilisearch is quite old, and we have made some improvements since then. Could you please try using the latest version? It might fix the issue for you.

[hieunv5501]

Hello @hieunv5501. Sorry for that. It could be an issue with the Japanese tokenizer. However, your version of meilisearch is quite old, and we have made some improvements since then. Could you please try using the latest version? It might fix the issue for you.

I tried using v1.1.1 version but still the problem is not solved.

[ManyTheFish]

Short version (@oluademola): it's a bug coming from uncertainty in the Language detection when no Hiragana/katakana characters are present in the query. (related to #3565)

Hello @hieunv5501,
Your issue seems related to #3565, we found out that if your dataset contains several documents without hiragana or katana characters, then Meilisearch struggles to make the difference between Chinese and Japanese when the search query only contains kanjis characters.
So far we didn't completely fix the issue, however, there is a prototype that deactivates Chinese Language detection forcing Meilisearch to always consider CJ characters as Japanese characters, this prototype is up to date with the last stable Meilisearch version so don't hesitate to use it:
meilisearch/product#532 (comment)

In addition to that, I want to make you know that we have a dedicated product discussion to speak about Japanese Language specificities and how Meilisearch could improve in supporting it, we have some really active contributors here, so don't hesitate to follow the discussion, ask questions or even suggest possible improvements!

meilisearch/product#532

Thank you for your report!

[curquiza]

Thank you Many for the detail answer, closing this issue in favor of the issues and discussions Many linked
Let us know @hieunv5501 if we were wrong and it does not fix your issue