Skip to content

Commit

Permalink
Merge #108
Browse files Browse the repository at this point in the history
108: Update Hebrew segmenter link to unicode-segmentation instead of Jieba r=Kerollmops a=ManyTheFish

Update Hebrew segmenter link to unicode-segmentation instead of Jieba

fix #107


Co-authored-by: Many the fish <many@meilisearch.com>
  • Loading branch information
bors[bot] and ManyTheFish committed Jun 9, 2022
2 parents 1b274cd + 8346aa2 commit 82c9f3b
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions README.md
Expand Up @@ -18,7 +18,7 @@ Charabia provides a simple API to segment, normalize, or tokenize (segment + nor
|---------------------|-------------------------------------------------------------------------------|---------------------------|-------------------|---|
| **Latin** - **Any** |[unicode-segmentation](https://github.com/unicode-rs/unicode-segmentation) | ✅ lowercase + deunicode | 🟨 ~12MiB/sec | 🟧 ~5MiB/sec |
| **Chinese** - **CMN** 🇨🇳 |[jieba](https://github.com/messense/jieba-rs) | ✅ traditional-to-simplified conversion | 🟨 ~9MiB/sec | 🟧 ~4MiB/sec |
| **Hebrew** 🇮🇱 |[unicode-segmentation](https://github.com/messense/jieba-rs) | ✅ diacritics removal | 🟩 ~21MiB/sec | 🟨 ~9MiB/sec |
| **Hebrew** 🇮🇱 |[unicode-segmentation](https://github.com/unicode-rs/unicode-segmentation) | ✅ diacritics removal | 🟩 ~21MiB/sec | 🟨 ~9MiB/sec |
| **Japanese** 🇯🇵 |[lindera](https://github.com/lindera-morphology/lindera) || 🟧 ~3MiB/sec | 🟧 ~3MiB/sec |

We aim to provide global language support, and your feedback helps us [move closer to that goal](https://docs.meilisearch.com/learn/advanced/language.html#improving-our-language-support). If you notice inconsistencies in your search results or the way your documents are processed, please open an issue on our [GitHub repository](https://github.com/meilisearch/charabia/issues/new/choose).
Expand Down Expand Up @@ -72,4 +72,4 @@ let mut segments = orig.segment_str();
assert_eq!(segments.next(), Some("The"));
assert_eq!(segments.next(), Some(" "));
assert_eq!(segments.next(), Some("quick"));
```
```

0 comments on commit 82c9f3b

Please sign in to comment.