You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hello,
it looks like that chinese in a text breaks multi language detection. I know: its experimental, but it works most of the time pretty good.
Example:
`text="Płaszczowo-rurowe wymienniki ciepła Uszczelkowe der blaue himmel über berlin 中文 the quick brown fox jumps over the lazy dog"
Hi Lutz, thank you for your report. I will try to improve the multi language detection algorithm in future releases. Your example might help in this respect.
It turns out that the cause of this issue is the same as for the issue #154 which I've just fixed with commit 67fdebc.
The output of your code after the fix is:
[
DetectionResult(start_index=0, end_index=48, word_count=4, language=Language.POLISH),
DetectionResult(start_index=48, end_index=80, word_count=7, language=Language.GERMAN),
DetectionResult(start_index=80, end_index=123, word_count=9, language=Language.ENGLISH)
]
POLISH Płaszczowo-rurowe wymienniki ciepła Uszczelkowe
GERMAN der blaue himmel über berlin 中文
ENGLISH the quick brown fox jumps over the lazy dog
Hello,
it looks like that chinese in a text breaks multi language detection. I know: its experimental, but it works most of the time pretty good.
Example:
`text="Płaszczowo-rurowe wymienniki ciepła Uszczelkowe der blaue himmel über berlin 中文 the quick brown fox jumps over the lazy dog"
detector=LanguageDetectorBuilder.from_languages(Language.ENGLISH, Language.GERMAN, Language.POLISH).build()
detector.detect_multiple_languages_of(text)
[DetectionResult(start_index=0, end_index=48, word_count=4, language=Language.POLISH), DetectionResult(start_index=48, end_index=77, word_count=5, language=Language.GERMAN)]
`
The text was updated successfully, but these errors were encountered: