Case-changing Twig filters and slug generation aren't language-aware (Turkish İ becomes "i" + U+0307)

[thisisjamessmith]

A little unsure whether to label this as a bug or feature request, so kept it to Discussions for now...

Description

Twig's |title, |lower, |upper and |capitalize filters, and Craft's slug generation, all use mbstring's default Unicode case mappings regardless of the site language. For Turkish (and Azerbaijani, Lithuanian and Greek) those mappings are wrong.

The most visible symptom: lowercasing the dotted capital İ (U+0130) yields a plain i followed by COMBINING DOT ABOVE (U+0307). The browser stacks the second dot on the i's own dot, so the letter renders noticeably darker and heavier than its
neighbours. Editors report it as "the i's are bold".

Twig itself has no notion of a current language, so this is reasonable behaviour for Twig core. Craft does know the language (Craft::$app->language), already overrides a number of core Twig filters in craft\web\twig\Extension, and already
requires the intl extension, which ships language-aware casing via ICU.

Steps to reproduce

1. Create a site whose language is tr (Turkish).
2. Create an entry titled SİNTERLENMİŞ PLAKA in that site.
3. In a template, output {{ entry.title|title }}.
4. Save an entry titled Çağdaş İlham in that site and look at the generated slug.

Expected

• Step 3 outputs Sinterlenmiş Plaka.
• Step 4 generates çağdaş-ilham.

Actual

• Step 3 outputs Si̇nterlenmi̇ş Plaka, which renders with a doubled dot on both i's.
• Step 4 generates çağdaş-i̇lham with a combining dot inside the URI, because ElementHelper::normalizeSlug() calls mb_strtolower() and the subsequent character filter deliberately keeps \p{M}.

|upper has the mirror-image fault: {{ 'ılık'|upper }} should be ILIK and {{ 'izmir'|upper }} should be İZMİR on a Turkish site.

Proposed solution

Make the casing filters and StringHelper casing helpers language-aware when ICU has rules for the current language, and fall back to the current behaviour otherwise so every other language is unaffected:

$lang = strtolower(explode('-', Craft::$app->language)[0]);
$transliterator = Transliterator::create("$lang-Title"); // also "-Lower", "-Upper"
if ($transliterator) {
    $string = Normalizer::normalize($string, Normalizer::FORM_C);
    return $transliterator->transliterate($string);
}
return CoreExtension::titleCase($charset, $string);

ICU returns null for languages without special rules (en, fr, es, de…), so the fallback keeps them byte-for-byte identical to today. The NFC normalisation step handles content where İ was typed or imported as I + U+0307.

Slug generation could use the same lowercasing path, or NFC-normalise before filtering, so the combining mark can't survive into a URI.

Workaround

Register a Twig extension that overrides the four filters with the above, via Craft::$app->getView()->registerSiteTwigExtension(). That's what we've done on a Craft 4.17 site; it works but every Turkish-language Craft site would need it.

Additional info

• Craft version: reproduced on 4.17.11; the code paths are unchanged in 5.10.10
• PHP version: 8.3
• Database driver & version: MySQL 8
• Related: [Bug] Title filter twigphp/Twig#2596 (Greek final sigma) is the same class of fault in
  Twig's |title, which Twig is unlikely to fix without a locale concept.

[brandonkelly]

Thanks for pointing that out! I’ve made those casing filters and PHP-based slug generation language-aware for Craft 5.12, as well as the next Craft 6 alpha release (#19558).

[thisisjamessmith]

wow so fast, thanks Brandon!