Unicode sorting for music folder "broken"

[netplumber]

Unicode sorting for music folder "broken". Sorting for folders with "latin" names is fine.
Build - Logitech Media Server Version: 7.9.2 - 1533033975

[dolodobendan]

Never noticed this before, but you're right. In Web UI (and iPeng):

Home > Music Folder > FLACs

Haydn, Joseph
Hindemith, Paul
Händel, Georg Friedrich
Holst, Gustav

LMS 7.9.2 - 1533559127

[mherger]

FWIW: "regular" browsing is using SQLite's sorting routines which comes with extensive ICU (https://en.wikipedia.org/wiki/International_Components_for_Unicode) support. BMF isn't using SQLite. We'd have to somehow implement ICU support for this separately.

[mherger]

...or we use a temporary SQLite table rather than internal data structures to handle this. Just an idea. Not saying I'm going to investigate this any time soon. Or not. Whatever. :-D

[kimmot]

008_bmf_sort_fix_v2.patch.txt
This is a patch I have used in Daphile to fix BMF sorting issue already for quite a while. I really don't how did I ended up into this but it worked. Daphile is using UTF-8.

[michaelherger]

Your patch wouldn't really fix the sorting, as it's much more complicated than that. It needs to take locale into account. Eg. in German Hand < Händel < Haydn. But in English Hand < Haydn < Händel.

[kimmot]

009_env_var_for_lms_sort_lang.patch.txt
Maybe this patch was the other part of the fix that was required. The sorting locale is selected in Daphile with the environment variable "LMS_COLLATE" (quite a hack, sorry). But I'm pretty sure I tested the BMF sorting is working correctly in Daphile taking into account the sorting rules of different locales.

[kimmot]

I had to test it again to be sure. I think those two patches fixed it in Daphile. The LMS_COLLATE environment variable is defined in Daphile Settings.

[mherger]

Hmm... interesting. The second patch must be for something else, as it's dealing with SQLite - which we're not using for BMF. But I looked again at the code around the first patch. It does indeed take collation into account in some way. I'll play with that. Thanks a lot!

[kimmot]

Sorry... I really didn't remember anymore what's the reason for second patch 😂️. I though it was relevant for this but it's actually taking care that LMS sorting is done according to Daphile setting even if the user has selected different language in LMS settings.

[kimmot]

One more detail. It's important to set LC_CTYPE to corrent sorting locale because LMS switches it temporarily to LC_COLLATE in Slim/Music/Info.pm sortFilename. See here:
https://github.com/Logitech/slimserver/blob/public/8.4/Slim/Music/Info.pm#L957

[michaelherger]

Ok, things are a little complicated 😁. Your patch indeed does work as expected on Linux. But it breaks sorting on Windows... and macOS wouldn't sort correctly using LC_COLLATE at all. This seems to be a popular pain point. Searching for "macOS sort LC_COLLATE" you'd find lots of discussions about the different behavior compared to Linux and other unix systems.

I'm going to split this up per OS: keep existing method for Windows, re-write using SQLite's collation for macOS, use your method for all other systems...