FR: Locale base string comparison for collection sort

[FrittenKeeZ]

Default behaviour of collection sort is to use strcasecmp(), which is great for English but bad for every other language.
This changes the default to use the Collator class if the Intl extension is present, or use current behaviour as fallback.

In Statamic\API\Helper change line 341 from:

return strcasecmp($one, $two);

To:

return Str::compareCase($one, $two);

In Statamic\API\Str add the following code:

/**
 * Locale based string comparison.
 *
 * If Intl extension is present, Collator::compare() is used,
 * otherwise strcmp() is used.
 *
 * @param  string  $str1
 * @param  string  $str2
 * @param  string|null  $locale  Will use default locale if NULL is provided.
 * @return int|false  Return comparison result, or FALSE on error.
 */
public static function compare($str1, $str2, $locale = null)
{
    if (class_exists('Collator')) {
        static $collators = [];

        if (! $locale) {
            $locale = app()->getLocale();
        }
        if (! isset($collators[$locale])) {
            $collators[$locale] = new \Collator($locale);
        }
        return $collators[$locale]->compare($str1, $str2);
    }

    return strcmp($str1, $str2);
}

/**
 * Locale based case-insensitive string comparison.
 *
 * If Intl extension is present, Collator::compare() is used,
 * otherwise strcmp() is used.
 *
 * @see Str::compare() for params and return.
 */
public static function compareCase($str1, $str2, $locale = null)
{
    return self::compare(self::lower($str1), self::lower($str2), $locale);
}

All you then need to do is set the current locale and ensure the Intl extension is installed.

[rrelmy]

Nice work 👍

[jasonvarga]

Can you give me an example of a list of titles/words and how they should be sorted in a particular language?

[FrittenKeeZ]

Sure... given this list of Danish cities in a random order:

• Billund
• Aalborg
• Frederiksberg
• Ølstykke
• Ringsted
• Allinge
• Ærøskøbing

This is the natural sort in Danish:

• Allinge
• Billund
• Frederiksberg
• Ringsted
• Ærøskøbing
• Ølstykke
• Aalborg

ÆØÅ is the last 3 letters in Danish, and Å is the new way of writing Aa, though cities use the old way.
Also, the reason why I didn't use strcoll() is that you can't be sure that a server has locale installed, rendering that function useless.

[jasonvarga]

Thanks!

[anvart]

Just for those who is still waiting for the feature but does not want to modify core files: here is my code for a custom modifier that does the sorting right:

public function index($value, $params, $context)
{
    $originalCollateLocale = setlocale(LC_COLLATE, 0);
    $currentCollateLocale = Config::getFullLocale();

    $key = array_get($params, 0);
    $is_descending = strtolower(array_get($params, 1)) == 'desc';

    if ($key === 'random') {
        return $this->shuffle($value);
    }

    // Using sort="true" will allow primitive arrays to be sorted.
    if ($key === 'true') {
        natcasesort($value);
        return $is_descending ? $this->reverse($value) : $value;
    }
    setlocale(LC_COLLATE, $currentCollateLocale);
    $return = collect($value)->sortBy($key, SORT_LOCALE_STRING, $is_descending)->values()->toArray();
    setlocale(LC_COLLATE, $originalCollateLocale);

    return $return;
}

Please make sure to set full locale name in system.yaml.

locales:
  de:
    full: de_CH.utf8
    ...

Also I had a problem with testing the code because in my local setup (MacOS/AMPPS) some locale-related modules are missing. Still no idea how to solve it. Works as expected on the hosting.

P.s. I'm not sure if it was a mauvais ton to write into a closed issue, please correct me.

[jasonvarga]

It was added in September 2017. What's not working?

[anvart]

Ah, now I see where the confusion comes from. The original claim was about collection sorting and I am talking about sort modifier and these two are different things.

Original BaseModifiers.php:sort() calls Collection::sortBy() which does this:

# $options = SORT_REGULAR
$descending ? arsort($results, $options) : asort($results, $options);

And asort/arsort with SORT_REGULAR do not work with non-English strings properly.

That's why I came to idea of the plugin.