New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FR: Locale base string comparison for collection sort #1574

Closed
FrittenKeeZ opened this Issue Aug 30, 2017 · 7 comments

Comments

Projects
None yet
4 participants
@FrittenKeeZ

FrittenKeeZ commented Aug 30, 2017

Default behaviour of collection sort is to use strcasecmp(), which is great for English but bad for every other language.
This changes the default to use the Collator class if the Intl extension is present, or use current behaviour as fallback.

In Statamic\API\Helper change line 341 from:

return strcasecmp($one, $two);

To:

return Str::compareCase($one, $two);

In Statamic\API\Str add the following code:

/**
 * Locale based string comparison.
 *
 * If Intl extension is present, Collator::compare() is used,
 * otherwise strcmp() is used.
 *
 * @param  string  $str1
 * @param  string  $str2
 * @param  string|null  $locale  Will use default locale if NULL is provided.
 * @return int|false  Return comparison result, or FALSE on error.
 */
public static function compare($str1, $str2, $locale = null)
{
    if (class_exists('Collator')) {
        static $collators = [];

        if (! $locale) {
            $locale = app()->getLocale();
        }
        if (! isset($collators[$locale])) {
            $collators[$locale] = new \Collator($locale);
        }
        return $collators[$locale]->compare($str1, $str2);
    }

    return strcmp($str1, $str2);
}

/**
 * Locale based case-insensitive string comparison.
 *
 * If Intl extension is present, Collator::compare() is used,
 * otherwise strcmp() is used.
 *
 * @see Str::compare() for params and return.
 */
public static function compareCase($str1, $str2, $locale = null)
{
    return self::compare(self::lower($str1), self::lower($str2), $locale);
}

All you then need to do is set the current locale and ensure the Intl extension is installed.

@rrelmy

This comment has been minimized.

rrelmy commented Aug 30, 2017

Nice work 👍

@jasonvarga

This comment has been minimized.

Member

jasonvarga commented Aug 30, 2017

Can you give me an example of a list of titles/words and how they should be sorted in a particular language?

@FrittenKeeZ

This comment has been minimized.

FrittenKeeZ commented Aug 30, 2017

Sure... given this list of Danish cities in a random order:

  • Billund
  • Aalborg
  • Frederiksberg
  • Ølstykke
  • Ringsted
  • Allinge
  • Ærøskøbing

This is the natural sort in Danish:

  • Allinge
  • Billund
  • Frederiksberg
  • Ringsted
  • Ærøskøbing
  • Ølstykke
  • Aalborg

ÆØÅ is the last 3 letters in Danish, and Å is the new way of writing Aa, though cities use the old way.
Also, the reason why I didn't use strcoll() is that you can't be sure that a server has locale installed, rendering that function useless.

@jasonvarga

This comment has been minimized.

Member

jasonvarga commented Aug 30, 2017

Thanks!

@anvart

This comment has been minimized.

anvart commented Nov 19, 2018

Just for those who is still waiting for the feature but does not want to modify core files: here is my code for a custom modifier that does the sorting right:

public function index($value, $params, $context)
{
    $originalCollateLocale = setlocale(LC_COLLATE, 0);
    $currentCollateLocale = Config::getFullLocale();

    $key = array_get($params, 0);
    $is_descending = strtolower(array_get($params, 1)) == 'desc';

    if ($key === 'random') {
        return $this->shuffle($value);
    }

    // Using sort="true" will allow primitive arrays to be sorted.
    if ($key === 'true') {
        natcasesort($value);
        return $is_descending ? $this->reverse($value) : $value;
    }
    setlocale(LC_COLLATE, $currentCollateLocale);
    $return = collect($value)->sortBy($key, SORT_LOCALE_STRING, $is_descending)->values()->toArray();
    setlocale(LC_COLLATE, $originalCollateLocale);

    return $return;
}

Please make sure to set full locale name in system.yaml.

locales:
  de:
    full: de_CH.utf8
    ...

Also I had a problem with testing the code because in my local setup (MacOS/AMPPS) some locale-related modules are missing. Still no idea how to solve it. Works as expected on the hosting.

P.s. I'm not sure if it was a mauvais ton to write into a closed issue, please correct me.

@jasonvarga

This comment has been minimized.

Member

jasonvarga commented Nov 21, 2018

It was added in September 2017. What's not working?

@anvart

This comment has been minimized.

anvart commented Nov 21, 2018

Ah, now I see where the confusion comes from. The original claim was about collection sorting and I am talking about sort modifier and these two are different things.

Original BaseModifiers.php:sort() calls Collection::sortBy() which does this:

# $options = SORT_REGULAR
$descending ? arsort($results, $options) : asort($results, $options);

And asort/arsort with SORT_REGULAR do not work with non-English strings properly.

That's why I came to idea of the plugin.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment