Most frequent alphabet detection causes inaccuracies when multiple alphabets have same occurrence count

[Marcono1234]

The mostFrequentAlphabet detection of filterLanguagesByRules uses maxByOrNull:

lingua/src/main/kotlin/com/github/pemistahl/lingua/api/LanguageDetector.kt

Line 318 in 7e415ae

val mostFrequentAlphabet = detectedAlphabets.entries.maxByOrNull { it.value }!!.key

When text now contains words of multiple alphabets which have the same occurence count, maxByOrNull would pick one of them as most frequent one.
For example the following will return only Greek with 1.0 confidence:

LanguageDetectorBuilder.fromLanguages(Language.GREEK, Language.BENGALI).build()
  .computeLanguageConfidenceValues("Παπασταθόπουλου ভয়াবহ")

When instead getting a set of alphabets with maximum count the result is (roughly): {GREEK=1.0, BENGALI=0.6349401470311586}

[Marcono1234]

Or maybe in general it would be good to adjust the rule based detection to not make rash decisions. For example when a text is half Japanese and half English (with the English part being the translation), Lingua will most likely return only Japanese with a confidence of 1.0.

Edit: Though maybe this is actually a feature request asking for detection of multiple languages in a text similar to #38, except without having to know where the sections in different languages are, approximate precentages might suffice.

[pemistahl]

This problem has been fixed now.