Handle Japanese by setting up Lindera tokenizer

[curquiza]

In progress by the community here:

• [WIP] Setup lindera tokenizer for ko/ja support charabia#49
• Setup lindera tokenizer for ja support ( related with #49 ) charabia#70

If there is no answer from the community, the work should be finished to be integrated to v0.27.0

⚠️ This version of our tokenizer will NOT use whatlang::detect_lang() since we don't know the impact on the performance. For this first implementation, we will use only whatlang::detect_script() to detect the script and use Lindera accordingly or not.
The whatlang::detect_script() will not be perfectly efficient since Japanese documents can be detected as Mandarin script, and in this situation, our tokenizer will use Jieba tokenizer instead of Lindera.
A usage of whatlang::detect_lang() can be considered in the future but will need benchmarks to avoid any loss of performance.

Should also fix #2159

---

Steps

• Merge Setup lindera tokenizer for ja support ( related with #49 ) charabia#70
• Release tokenizer
• Bump tokenizer dependency in Milli
• Release Milli with the new tokenizer dependency
• Bump milli in Meilisearch

[curquiza]

@meilisearch/docs-team, I don't know if you have a page where you listed the supported languages.

[Northn]

@meilisearch/docs-team, I don't know if you have a page where you listed the supported languages.

https://docs.meilisearch.com/learn/what_is_meilisearch/language.html

[curquiza]

Milli was bumped in #2244, with milli v0.24.0 containing the new version of the tokenizer. We can close this issue. The change will be effective in v0.27.0.

[curquiza]

We have an issue with Lindera compilation. I re-open this issue until we find out a fix! @Kerollmops is on it

[Kerollmops]

We are awaiting the next release of milli, the version after v0.24.0 which contains meilisearch/milli#475 that uses the latest version of lindera with lindera-morphology/lindera#164.

There were two issues:

1. The lindera reqwest version was colliding with our own version and,
2. lindera was downloading its dictionary from Google Drive and it was failing very often, breaking our builds, they switched to Sourceforge, which is more stable now.

[Kerollmops]

We bumped milli to v0.24.1 in #2280 to close this issue.