Russian Alphabet Sorting - "Ë" Letter

[casalsgh]

Issue Description
-When sorted alphabetically, contacts with the letter "Ё" are located at the bottom of the list but should be after the letter "E". The letter "Ё" is the seventh in the Russian alphabet, but in the sorting of contacts is the 33rd place.

Steps to Reproduce
-create a user with the Russian letter "Ё" at the beginning and a few other users with a name consisting of Russian letters
-Try to search for the user in the contact list

Expected Behavior
"Ё" is the seventh in the Russian alphabet therefore it should not be at the end of the search result

Actual Behavior
The name with this letter was at the end of the search result

[casalsgh]

@hardikbhatia777 is this something you might want to be assigned?

[hardikbhatia777]

@hardikbhatia777 is this something you might want to be assigned?

Yes. I'll start working on it right away.

[hardikbhatia777]

@casalsgh I have tried many combinations of Russian words. Lexicographically, the name starting with Ё is appearing as the first result regardless in the directory for me, while the other names seem to switch positions and arrange in the correct order:

Is this the issue you were talking about? As far as I can tell, it should be
Автомобиль
Город
Ёж
Медведь
Яблоко

EDIT: I see what you meant now, I'm working on a fix.

[rituraj12797]

working on it

[hardikbhatia777]

Hello @casalsgh, after a lot of research, this seems to be a very unique issue with UNICODE itself rather than the sorting function being used. I found that "Ë" is treated as follows:

As such, I've coded a new sorting function specifically for this case using Intl.collator, which supports non-unicode sorting through language-sensitive string comparison. I have hard-coded it into the table component itself for now just to show the working.

This is a workaround which would require refactoring of quite a bit of code, or more practically, maintaining a state to see if sorting occurs using 'name' and using the collator object in those specific cases.

In short,

1. This issue can simply be ignored, since it occurs due to UNICODE being the way it is (which can change in the future).
2. The whole sorting feature can be refactored to convert all sorting logic to Intl.collator objects.
3. A state can be maintained to specifically check name sorting and use collator object in those specific cases.

I'm unsure of which approach you would consider best, hence I thought it would be best to reach out first and ask.
Please let me know which solution you would prefer and I'll implement that right away :)

Best Regards,
Hardik

[casalsgh]

Hardik many thanks for the analyzes. Will check with others on the internal team and get back to you shortly.

[casalsgh]

@hardikbhatia777 do you mind checking the issue in the mobile app for Rocket.Chat?
Seems there is already a solution that mobile side developed. Maybe something that can just be checked and replicated for the main repo.
BTW - an engineer also shared this article, seems interesting (was informative for me, that is why I'm sharing here)-> https://tonsky.me/blog/unicode/

[hardikbhatia777]

@hardikbhatia777 do you mind checking the issue in the mobile app for Rocket.Chat? Seems there is already a solution that mobile side developed. Maybe something that can just be checked and replicated for the main repo. BTW - an engineer also shared this article, seems interesting (was informative for me, that is why I'm sharing here)-> https://tonsky.me/blog/unicode/

Great read! Thank you for sharing.

I'll check out the mobile repo and get back to you very soon.

[hardikbhatia777]

@casalsgh I went through the mobile repository, but I wasn't able to locate the specific sorting function that was fixing the issue. Is there any way you could direct me to the file?

[casalsgh]

@hardikbhatia777 based on your assessment and discovery so far pointing to Unicode issues + complexity/level of effort to have it adjusted for rocket.chat we will not be pursuing the ticket. Will be tagged as won't fix and closed.
You were the key contributor helping with analyses that lead to this decision, so thank you very much for your efforts.

PS: I'll tag you on future tickets again as I mentioned I would get you assigned for items.